You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by pg...@apache.org on 2002/10/04 10:17:50 UTC
cvs commit: jakarta-james/src/java/org/apache/james/mailrepository JDBCMailRepository.java
pgoldstein 2002/10/04 01:17:50
Modified: src/java/org/apache/james/util/mordred JdbcDataSource.java
src/java/org/apache/james/nntpserver NNTPHandler.java
src/java/org/apache/james/nntpserver/repository
ArticleIDRepository.java NNTPArticleImpl.java
NNTPGroup.java NNTPGroupImpl.java
NNTPRepositoryImpl.java NNTPSpooler.java
src/java/org/apache/james/userrepository
AbstractJdbcUsersRepository.java
src/java/org/apache/james/mailrepository
JDBCMailRepository.java
Removed: src/java/org/apache/james/nntpserver LISTGroup.java
src/java/org/apache/james/nntpserver/repository
NNTPUtil.java
Log:
Rearranged object encapsulation for several classes in the NNTP server code
Fixed a critical threading issue (race condition on creating new articles)
Reduced group object creation as part of the fix for the threading issue.
Moved file URL lookup code into a single location.
Removed extraneous classes to reduce conceptual weight of the NNTP code.
Revision Changes Path
1.17 +2 -6 jakarta-james/src/java/org/apache/james/util/mordred/JdbcDataSource.java
Index: JdbcDataSource.java
===================================================================
RCS file: /home/cvs/jakarta-james/src/java/org/apache/james/util/mordred/JdbcDataSource.java,v
retrieving revision 1.16
retrieving revision 1.17
diff -u -r1.16 -r1.17
--- JdbcDataSource.java 27 Sep 2002 07:48:40 -0000 1.16
+++ JdbcDataSource.java 4 Oct 2002 08:17:49 -0000 1.17
@@ -209,11 +209,7 @@
}
/**
- * Configure and set up DB connection. Here we set the connection information needed to create
- * the Connection objects.
- *
- * @param configuration The Configuration object needed to describe the connection.
- * @throws ConfigurationException
+ * @see org.apache.avalon.framework.configuration.Configurable#configure(Configuration)
*/
public void configure(final Configuration configuration)
throws ConfigurationException {
1.18 +35 -27 jakarta-james/src/java/org/apache/james/nntpserver/NNTPHandler.java
Index: NNTPHandler.java
===================================================================
RCS file: /home/cvs/jakarta-james/src/java/org/apache/james/nntpserver/NNTPHandler.java,v
retrieving revision 1.17
retrieving revision 1.18
diff -u -r1.17 -r1.18
--- NNTPHandler.java 2 Oct 2002 09:53:38 -0000 1.17
+++ NNTPHandler.java 4 Oct 2002 08:17:49 -0000 1.18
@@ -487,7 +487,7 @@
// see section 9.4.1
String wildmat = "*";
- LISTGroup output = null;
+ boolean isListNewsgroups = false;
String extension = argument;
if (argument != null) {
@@ -499,36 +499,44 @@
extension = extension.toUpperCase(Locale.US);
}
- if ((extension == null) || (extension.equals("ACTIVE"))) {
- output = LISTGroup.Factory.ACTIVE(writer);
- } else if (extension.equals("NEWSGROUPS") ) {
- output = LISTGroup.Factory.NEWSGROUPS(writer);
- } else if (extension.equals("EXTENSIONS") ) {
- doLISTEXTENSIONS();
- return;
- } else if (extension.equals("OVERVIEW.FMT") ) {
- doLISTOVERVIEWFMT();
- return;
- } else if (extension.equals("ACTIVE.TIMES") ) {
- // not supported - 9.4.2.1, 9.4.3.1, 9.4.4.1
- writer.println("503 program error, function not performed");
- return;
- } else if (extension.equals("DISTRIBUTIONS") ) {
- // not supported - 9.4.2.1, 9.4.3.1, 9.4.4.1
- writer.println("503 program error, function not performed");
- return;
- } else if (extension.equals("DISTRIB.PATS") ) {
- // not supported - 9.4.2.1, 9.4.3.1, 9.4.4.1
- writer.println("503 program error, function not performed");
- return;
- } else {
- writer.println("501 Syntax error");
- return;
+ if (extension != null) {
+ if (extension.equals("ACTIVE")) {
+ isListNewsgroups = false;
+ } else if (extension.equals("NEWSGROUPS") ) {
+ isListNewsgroups = true;
+ } else if (extension.equals("EXTENSIONS") ) {
+ doLISTEXTENSIONS();
+ return;
+ } else if (extension.equals("OVERVIEW.FMT") ) {
+ doLISTOVERVIEWFMT();
+ return;
+ } else if (extension.equals("ACTIVE.TIMES") ) {
+ // not supported - 9.4.2.1, 9.4.3.1, 9.4.4.1
+ writer.println("503 program error, function not performed");
+ return;
+ } else if (extension.equals("DISTRIBUTIONS") ) {
+ // not supported - 9.4.2.1, 9.4.3.1, 9.4.4.1
+ writer.println("503 program error, function not performed");
+ return;
+ } else if (extension.equals("DISTRIB.PATS") ) {
+ // not supported - 9.4.2.1, 9.4.3.1, 9.4.4.1
+ writer.println("503 program error, function not performed");
+ return;
+ } else {
+ writer.println("501 Syntax error");
+ return;
+ }
}
+
Iterator iter = repo.getMatchedGroups(wildmat);
writer.println("215 list of newsgroups follows");
while ( iter.hasNext() ) {
- output.show((NNTPGroup)iter.next());
+ NNTPGroup theGroup = (NNTPGroup)iter.next();
+ if (isListNewsgroups) {
+ writer.println(theGroup.getListNewsgroupsFormat());
+ } else {
+ writer.println(theGroup.getListFormat());
+ }
}
writer.println(".");
}
1.9 +0 -9 jakarta-james/src/java/org/apache/james/nntpserver/repository/ArticleIDRepository.java
Index: ArticleIDRepository.java
===================================================================
RCS file: /home/cvs/jakarta-james/src/java/org/apache/james/nntpserver/repository/ArticleIDRepository.java,v
retrieving revision 1.8
retrieving revision 1.9
diff -u -r1.8 -r1.9
--- ArticleIDRepository.java 2 Oct 2002 09:53:38 -0000 1.8
+++ ArticleIDRepository.java 4 Oct 2002 08:17:49 -0000 1.9
@@ -55,15 +55,6 @@
}
/**
- * Returns the root of the repository.
- *
- * @return the root of the repository
- */
- public File getPath() {
- return root;
- }
-
- /**
* Generate a new article ID for use in the repository.
*/
String generateArticleID() {
1.8 +9 -2 jakarta-james/src/java/org/apache/james/nntpserver/repository/NNTPArticleImpl.java
Index: NNTPArticleImpl.java
===================================================================
RCS file: /home/cvs/jakarta-james/src/java/org/apache/james/nntpserver/repository/NNTPArticleImpl.java,v
retrieving revision 1.7
retrieving revision 1.8
diff -u -r1.7 -r1.8
--- NNTPArticleImpl.java 2 Oct 2002 09:53:38 -0000 1.7
+++ NNTPArticleImpl.java 4 Oct 2002 08:17:49 -0000 1.8
@@ -25,19 +25,26 @@
private final File articleFile;
/**
+ * The newsgroup containing this article.
+ */
+ private final NNTPGroup group;
+
+ /**
* The sole constructor for this class.
*
+ * @param group the news group containing this article
* @param f the file that stores the article data
*/
- NNTPArticleImpl(File f) {
+ NNTPArticleImpl(NNTPGroup group, File f) {
articleFile = f;
+ this.group = group;
}
/**
* @see org.apache.james.nntpsever.repository.NNTPArticle#getGroup()
*/
public NNTPGroup getGroup() {
- return new NNTPGroupImpl(articleFile.getParentFile());
+ return group;
}
/**
1.4 +25 -3 jakarta-james/src/java/org/apache/james/nntpserver/repository/NNTPGroup.java
Index: NNTPGroup.java
===================================================================
RCS file: /home/cvs/jakarta-james/src/java/org/apache/james/nntpserver/repository/NNTPGroup.java,v
retrieving revision 1.3
retrieving revision 1.4
diff -u -r1.3 -r1.4
--- NNTPGroup.java 2 Oct 2002 09:53:38 -0000 1.3
+++ NNTPGroup.java 4 Oct 2002 08:17:49 -0000 1.4
@@ -7,6 +7,8 @@
*/
package org.apache.james.nntpserver.repository;
+import java.io.InputStream;
+import java.io.IOException;
import java.util.Date;
import java.util.Iterator;
@@ -109,8 +111,28 @@
Iterator getArticles();
/**
- * TODO: I don't understand the purpose of this method. It seems to
- * be an implementation hack.
+ * Retrieves the group information in a format consistent with
+ * a LIST or LIST ACTIVE return line
+ *
+ * @return the properly formatted string
+ */
+ String getListFormat();
+
+ /**
+ * Retrieves the group information in a format consistent with
+ * a LIST NEWSGROUPS return line
+ *
+ * @return the properly formatted string
+ */
+ String getListNewsgroupsFormat();
+
+ /**
+ * Adds an article to the group based on the data in the
+ * stream.
+ *
+ * @param newsStream the InputStream containing the article data
+ *
+ * @return the newly created article
*/
- Object getPath();
+ NNTPArticle addArticle(InputStream newsStream) throws IOException;
}
1.7 +67 -7 jakarta-james/src/java/org/apache/james/nntpserver/repository/NNTPGroupImpl.java
Index: NNTPGroupImpl.java
===================================================================
RCS file: /home/cvs/jakarta-james/src/java/org/apache/james/nntpserver/repository/NNTPGroupImpl.java,v
retrieving revision 1.6
retrieving revision 1.7
diff -u -r1.6 -r1.7
--- NNTPGroupImpl.java 2 Oct 2002 09:53:38 -0000 1.6
+++ NNTPGroupImpl.java 4 Oct 2002 08:17:49 -0000 1.7
@@ -10,9 +10,14 @@
import org.apache.avalon.excalibur.io.AndFileFilter;
import org.apache.avalon.excalibur.io.ExtensionFileFilter;
import org.apache.avalon.excalibur.io.InvertedFileFilter;
+import org.apache.avalon.excalibur.io.IOUtil;
+import org.apache.avalon.framework.logger.AbstractLogEnabled;
import org.apache.james.nntpserver.DateSinceFileFilter;
import java.io.File;
+import java.io.FileOutputStream;
+import java.io.InputStream;
+import java.io.IOException;
import java.util.ArrayList;
import java.util.Date;
import java.util.Iterator;
@@ -24,7 +29,7 @@
*
* @author Harmeet Bedi <ha...@kodemuse.com>
*/
-class NNTPGroupImpl implements NNTPGroup {
+class NNTPGroupImpl extends AbstractLogEnabled implements NNTPGroup {
/**
* The directory to which this group maps.
@@ -168,7 +173,7 @@
*/
public NNTPArticle getArticle(int number) {
File f = new File(root,number + "");
- return f.exists() ? new NNTPArticleImpl(f) : null;
+ return f.exists() ? new NNTPArticleImpl(this, f) : null;
}
/**
@@ -180,7 +185,7 @@
new InvertedFileFilter(new ExtensionFileFilter(".id"))));
List list = new ArrayList();
for ( int i = 0 ; i < f.length ; i++ )
- list.add(new NNTPArticleImpl(f[i]));
+ list.add(new NNTPArticleImpl(this, f[i]));
return list.iterator();
}
@@ -191,15 +196,70 @@
File[] f = root.listFiles();
List list = new ArrayList();
for ( int i = 0 ; i < f.length ; i++ )
- list.add(new NNTPArticleImpl(f[i]));
+ list.add(new NNTPArticleImpl(this, f[i]));
return list.iterator();
}
/**
- * @see org.apache.james.nntpserver.NNTPGroup#getPath()
+ * @see org.apache.james.nntpserver.repository.NNTPGroup#getListFormat()
*/
- public Object getPath() {
- return root;
+ public String getListFormat() {
+ StringBuffer showBuffer =
+ new StringBuffer(128)
+ .append(getName())
+ .append(" ")
+ .append(getFirstArticleNumber())
+ .append(" ")
+ .append(getLastArticleNumber())
+ .append(" ")
+ .append((isPostAllowed() ? "y":"n"));
+ return showBuffer.toString();
+ }
+
+ /**
+ * @see org.apache.james.nntpserver.repository.NNTPGroup#getListNewsgroupsFormat()
+ */
+ public String getListNewsgroupsFormat() {
+ StringBuffer showBuffer =
+ new StringBuffer(128)
+ .append(getName())
+ .append(" ")
+ .append(getDescription());
+ return showBuffer.toString();
+ }
+
+ /**
+ * @see org.apache.james.nntpserver.repository.NNTPGroup#addArticle(InputStream)
+ */
+ public NNTPArticle addArticle(InputStream newsStream)
+ throws IOException {
+ File articleFile = null;
+ synchronized (this) {
+ int artNum = getLastArticleNumber();
+ articleFile = new File(root,(artNum + 1)+"");
+ articleFile.createNewFile();
+ lastArticle++;
+ numOfArticles++;
+ }
+ if (getLogger().isDebugEnabled()) {
+ getLogger().debug("Copying message to: "+articleFile.getAbsolutePath());
+ }
+ FileOutputStream fout = null;
+ try {
+ fout = new FileOutputStream(articleFile);
+ IOUtil.copy(newsStream,fout);
+ fout.flush();
+ } finally {
+ try {
+ if (fout != null) {
+ fout.close();
+ }
+ } catch (IOException ioe) {
+ // Ignore this exception so we don't
+ // trash any "real" exceptions
+ }
+ }
+ return new NNTPArticleImpl(this, articleFile);
}
// public NNTPArticle getArticleFromID(String id) {
1.10 +139 -38 jakarta-james/src/java/org/apache/james/nntpserver/repository/NNTPRepositoryImpl.java
Index: NNTPRepositoryImpl.java
===================================================================
RCS file: /home/cvs/jakarta-james/src/java/org/apache/james/nntpserver/repository/NNTPRepositoryImpl.java,v
retrieving revision 1.9
retrieving revision 1.10
diff -u -r1.9 -r1.10
--- NNTPRepositoryImpl.java 2 Oct 2002 09:53:38 -0000 1.9
+++ NNTPRepositoryImpl.java 4 Oct 2002 08:17:49 -0000 1.10
@@ -17,6 +17,8 @@
import org.apache.avalon.framework.context.ContextException;
import org.apache.avalon.framework.context.Contextualizable;
import org.apache.avalon.framework.logger.AbstractLogEnabled;
+import org.apache.avalon.framework.logger.LogEnabled;
+import org.apache.james.context.AvalonContextUtilities;
import org.apache.james.nntpserver.DateSinceFileFilter;
import org.apache.james.nntpserver.NNTPException;
import org.apache.oro.io.GlobFilenameFilter;
@@ -27,6 +29,7 @@
import java.io.PrintStream;
import java.util.ArrayList;
import java.util.Date;
+import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
@@ -44,6 +47,11 @@
private Context context;
/**
+ * The configuration employed by this repository
+ */
+ private Configuration configuration;
+
+ /**
* Whether the repository is read only
*/
private boolean readOnly;
@@ -74,6 +82,33 @@
private String[] addGroups = null;
/**
+ * The root path as a String.
+ */
+ private String rootPathString = null;
+
+ /**
+ * The temp path as a String.
+ */
+ private String tempPathString = null;
+
+ /**
+ * The article ID path as a String.
+ */
+ private String articleIdPathString = null;
+
+ /**
+ * The domain suffix used for files in the article ID repository.
+ */
+ private String articleIDDomainSuffix = null;
+
+ /**
+ * This is a mapping of group names to NNTP group objects.
+ *
+ * TODO: This needs to be addressed so it scales better
+ */
+ private HashMap repositoryGroups = new HashMap();
+
+ /**
* @see org.apache.avalon.framework.context.Contextualizable#contextualize(Context)
*/
public void contextualize(Context context)
@@ -84,62 +119,70 @@
/**
* @see org.apache.avalon.framework.configuration.Configurable#configure(Configuration)
*/
- public void configure( Configuration configuration ) throws ConfigurationException {
- //System.out.println(getClass().getName() + ": configure");
- //NNTPUtil.show(configuration,System.out);
+ public void configure( Configuration aConfiguration ) throws ConfigurationException {
+ configuration = aConfiguration;
readOnly = configuration.getChild("readOnly").getValueAsBoolean(false);
- rootPath = NNTPUtil.getDirectory(context, configuration, "rootPath");
- tempPath = NNTPUtil.getDirectory(context, configuration, "tempPath");
- File articleIDPath = NNTPUtil.getDirectory(context, configuration, "articleIDPath");
- String articleIDDomainSuffix = configuration.getChild("articleIDDomainSuffix")
- .getValue("foo.bar.sho.boo");
- articleIDRepo = new ArticleIDRepository(articleIDPath,articleIDDomainSuffix);
- spool = (NNTPSpooler)NNTPUtil.createInstance
- (context,configuration.getChild("spool"),getLogger(),
- "org.apache.james.nntpserver.repository.NNTPSpooler");
- spool.setRepository(this);
- spool.setArticleIDRepository(articleIDRepo);
- if (getLogger().isDebugEnabled()) {
- getLogger().debug("repository:readOnly=" + readOnly);
- getLogger().debug("repository:rootPath=" + rootPath.getAbsolutePath());
- getLogger().debug("repository:tempPath=" + tempPath.getAbsolutePath());
- }
- configuration = configuration.getChild("newsgroups");
+ Configuration newsgroupConfiguration = configuration.getChild("newsgroups");
List addGroupsList = new ArrayList();
if ( configuration != null ) {
- Configuration[] children = configuration.getChildren("newsgroup");
- if ( children != null )
- for ( int i = 0 ; i < children.length ; i++ )
+ Configuration[] children = newsgroupConfiguration.getChildren("newsgroup");
+ if ( children != null ) {
+ for ( int i = 0 ; i < children.length ; i++ ) {
addGroupsList.add(children[i].getValue());
+ }
+ }
}
+ articleIDDomainSuffix = configuration.getChild("articleIDDomainSuffix")
+ .getValue("foo.bar.sho.boo");
addGroups = (String[])addGroupsList.toArray(new String[0]);
- getLogger().debug("repository configuration done");
+ rootPathString = configuration.getChild("rootPath").getValue();
+ tempPathString = configuration.getChild("tempPath").getValue();
+ articleIdPathString = configuration.getChild("articleIDPath").getValue();
+ getLogger().debug("Repository configuration done");
}
/**
* @see org.apache.avalon.framework.activity.Initializable#initialize()
*/
public void initialize() throws Exception {
- //System.out.println(getClass().getName() + ": init");
+
+ File articleIDPath = null;
+
+ try {
+ rootPath = AvalonContextUtilities.getFile(context, rootPathString);
+ tempPath = AvalonContextUtilities.getFile(context, tempPathString);
+ articleIDPath = AvalonContextUtilities.getFile(context, articleIdPathString);
+ } catch (Exception e) {
+ getLogger().fatalError(e.getMessage(), e);
+ throw e;
+ }
+
+ if ( articleIDPath.exists() == false ) {
+ articleIDPath.mkdirs();
+ }
+
+ articleIDRepo = new ArticleIDRepository(articleIDPath, articleIDDomainSuffix);
+ spool = (NNTPSpooler)createSpooler();
+ spool.setRepository(this);
+ spool.setArticleIDRepository(articleIDRepo);
+ if (getLogger().isDebugEnabled()) {
+ getLogger().debug("repository:readOnly=" + readOnly);
+ getLogger().debug("repository:rootPath=" + rootPath.getAbsolutePath());
+ getLogger().debug("repository:tempPath=" + tempPath.getAbsolutePath());
+ }
+
if ( rootPath.exists() == false ) {
rootPath.mkdirs();
}
for ( int i = 0 ; i < addGroups.length ; i++ ) {
- File groupF = new File(rootPath,addGroups[i]);
- if ( groupF.exists() == false ) {
- groupF.mkdirs();
+ File groupFile = new File(rootPath,addGroups[i]);
+ if ( groupFile.exists() == false ) {
+ groupFile.mkdirs();
}
}
if ( tempPath.exists() == false ) {
tempPath.mkdirs();
}
- File articleIDPath = articleIDRepo.getPath();
- if ( articleIDPath.exists() == false ) {
- articleIDPath.mkdirs();
- }
- if ( spool instanceof Initializable ) {
- ((Initializable)spool).initialize();
- }
getLogger().debug("repository initialization done");
}
@@ -154,8 +197,17 @@
* @see org.apache.james.nntpserver.repository.NNTPRepository#getGroup(String)
*/
public NNTPGroup getGroup(String groupName) {
- File f = new File(rootPath,groupName);
- return ( f.exists() && f.isDirectory() ) ? new NNTPGroupImpl(f) : null;
+ File groupFile = new File(rootPath,groupName);
+ NNTPGroup groupToReturn = null;
+ synchronized(this) {
+ groupToReturn = (NNTPGroup)repositoryGroups.get(groupName);
+ if ((groupToReturn == null) && groupFile.exists() && groupFile.isDirectory() ) {
+ groupToReturn = new NNTPGroupImpl(groupFile);
+ ((NNTPGroupImpl)groupToReturn).enableLogging(getLogger());
+ repositoryGroups.put(groupName, groupToReturn);
+ }
+ }
+ return groupToReturn;
}
/**
@@ -219,7 +271,9 @@
private Iterator getGroups(File[] f) {
List list = new ArrayList();
for ( int i = 0 ; i < f.length ; i++ ) {
- list.add(new NNTPGroupImpl(f[i]));
+ if (f[i] != null) {
+ list.add(getGroup(f[i].getName()));
+ }
}
return list.iterator();
}
@@ -273,4 +327,51 @@
}
};
}
+
+ /**
+ * Creates an instance of the spooler class.
+ *
+ * TODO: This method doesn't properly implement the Avalon lifecycle.
+ */
+ private NNTPSpooler createSpooler()
+ throws ConfigurationException {
+ String className = "org.apache.james.nntpserver.repository.NNTPSpooler";
+ Configuration spoolerConfiguration = configuration.getChild("spool");
+ try {
+ // Must be a subclass of org.apache.james.nntpserver.repository.NNTPSpooler
+ className = spoolerConfiguration.getAttribute("class");
+ } catch(ConfigurationException ce) {
+ // Use the default class.
+ }
+ try {
+ Object obj = getClass().getClassLoader().loadClass(className).newInstance();
+ // TODO: Need to support compose
+ if ( obj instanceof LogEnabled ) {
+ ((LogEnabled)obj).enableLogging( getLogger() );
+ }
+ if (obj instanceof Contextualizable) {
+ ((Contextualizable)obj).contextualize(context);
+ }
+ if ( obj instanceof Configurable ) {
+ ((Configurable)obj).configure(spoolerConfiguration.getChild("configuration"));
+ }
+ if ( obj instanceof Initializable ) {
+ ((Initializable)obj).initialize();
+ }
+ return (NNTPSpooler)obj;
+ } catch(ClassCastException cce) {
+ StringBuffer errorBuffer =
+ new StringBuffer(128)
+ .append("Spooler initialization failed because the spooler class ")
+ .append(className)
+ .append(" was not a subclass of org.apache.james.nntpserver.repository.NNTPSpooler");
+ String errorString = errorBuffer.toString();
+ getLogger().error(errorString, cce);
+ throw new ConfigurationException(errorString, cce);
+ } catch(Exception ex) {
+ getLogger().error("Spooler initialization failed",ex);
+ throw new ConfigurationException("Spooler initialization failed",ex);
+ }
+ }
+
}
1.9 +48 -46 jakarta-james/src/java/org/apache/james/nntpserver/repository/NNTPSpooler.java
Index: NNTPSpooler.java
===================================================================
RCS file: /home/cvs/jakarta-james/src/java/org/apache/james/nntpserver/repository/NNTPSpooler.java,v
retrieving revision 1.8
retrieving revision 1.9
diff -u -r1.8 -r1.9
--- NNTPSpooler.java 2 Oct 2002 09:53:38 -0000 1.8
+++ NNTPSpooler.java 4 Oct 2002 08:17:49 -0000 1.9
@@ -7,7 +7,6 @@
*/
package org.apache.james.nntpserver.repository;
-import org.apache.avalon.excalibur.io.IOUtil;
import org.apache.avalon.framework.activity.Initializable;
import org.apache.avalon.framework.configuration.Configurable;
import org.apache.avalon.framework.configuration.Configuration;
@@ -17,6 +16,7 @@
import org.apache.avalon.framework.context.Contextualizable;
import org.apache.avalon.framework.logger.AbstractLogEnabled;
import org.apache.avalon.framework.logger.LogEnabled;
+import org.apache.james.context.AvalonContextUtilities;
import org.apache.james.util.Lock;
import javax.mail.internet.MimeMessage;
@@ -50,6 +50,16 @@
private File spoolPath;
/**
+ * The String form of the spool directory.
+ */
+ private String spoolPathString;
+
+ /**
+ * The time the spooler threads sleep between processing
+ */
+ private int threadIdleTime = 0;
+
+ /**
* @see org.apache.avalon.framework.context.Contextualizable#contextualize(Context)
*/
public void contextualize(final Context context)
@@ -61,17 +71,36 @@
* @see org.apache.avalon.framework.configuration.Configurable#configure(Configuration)
*/
public void configure( Configuration configuration ) throws ConfigurationException {
- spoolPath = NNTPUtil.getDirectory(context, configuration, "spoolPath");
int threadCount = configuration.getChild("threadCount").getValueAsInteger(1);
- int threadIdleTime = configuration.getChild("threadIdleTime").getValueAsInteger(1000);
- //String tgName=configuration.getChild("threadGroupName").getValue("NNTPSpooler");
+ threadIdleTime = configuration.getChild("threadIdleTime").getValueAsInteger(1000);
+ spoolPathString = configuration.getChild("spoolPath").getValue();
worker = new SpoolerRunnable[threadCount];
+ }
+
+ /**
+ * @see org.apache.avalon.framework.activity.Initializable#initialize()
+ */
+ public void initialize() throws Exception {
+ //System.out.println(getClass().getName()+": init");
+
+ try {
+ spoolPath = AvalonContextUtilities.getFile(context, spoolPathString);
+ } catch (Exception e) {
+ getLogger().fatalError(e.getMessage(), e);
+ throw e;
+ }
+
for ( int i = 0 ; i < worker.length ; i++ ) {
worker[i] = new SpoolerRunnable(threadIdleTime,spoolPath);
if ( worker[i] instanceof LogEnabled ) {
((LogEnabled)worker[i]).enableLogging(getLogger());
}
}
+
+ // TODO: Replace this with a standard Avalon thread pool
+ for ( int i = 0 ; i < worker.length ; i++ ) {
+ new Thread(worker[i],"NNTPSpool-"+i).start();
+ }
}
/**
@@ -110,17 +139,6 @@
}
/**
- * @see org.apache.avalon.framework.activity.Initializable#initialize()
- */
- public void initialize() throws Exception {
- //System.out.println(getClass().getName()+": init");
- // TODO: Replace this with a standard Avalon thread pool
- for ( int i = 0 ; i < worker.length ; i++ ) {
- new Thread(worker[i],"NNTPSpool-"+i).start();
- }
- }
-
- /**
* A static inner class that provides the body for the spool
* threads.
*/
@@ -206,34 +224,34 @@
*
* @param f the spool file being processed
*/
- private void process(File f) throws Exception {
+ private void process(File spoolFile) throws Exception {
StringBuffer logBuffer =
new StringBuffer(160)
.append("process: ")
- .append(f.getAbsolutePath())
+ .append(spoolFile.getAbsolutePath())
.append(",")
- .append(f.getCanonicalPath());
+ .append(spoolFile.getCanonicalPath());
getLogger().debug(logBuffer.toString());
final MimeMessage msg;
String articleID;
// TODO: Why is this a block?
- { // get the message for copying to destination groups.
- FileInputStream fin = new FileInputStream(f);
+ { // Get the message for copying to destination groups.
+ FileInputStream fin = new FileInputStream(spoolFile);
msg = new MimeMessage(null,fin);
fin.close();
// ensure no duplicates exist.
String[] idheader = msg.getHeader("Message-Id");
- articleID = (idheader!=null && idheader.length>0?idheader[0]:null);
- if ( articleIDRepo.isExists(articleID) ) {
+ articleID = ((idheader != null && (idheader.length > 0))? idheader[0] : null);
+ if ((articleID != null) && ( articleIDRepo.isExists(articleID))) {
getLogger().debug("Message already exists: "+articleID);
- f.delete();
+ spoolFile.delete();
return;
}
if ( articleID == null ) {
articleID = articleIDRepo.generateArticleID();
msg.setHeader("Message-Id", articleID);
- FileOutputStream fout = new FileOutputStream(f);
+ FileOutputStream fout = new FileOutputStream(spoolFile);
msg.writeTo(fout);
fout.close();
}
@@ -245,34 +263,18 @@
getLogger().debug("Copying message to group: "+headers[i]);
NNTPGroup group = repo.getGroup(headers[i]);
if ( group == null ) {
- getLogger().debug("Group not found: "+headers[i]);
+ getLogger().error("Couldn't add article with article ID " + articleID + " to group " + headers[i] + " - group not found.");
continue;
}
- int artNum = group.getLastArticleNumber();
- // TODO: Encapsulate this in the NNTP group.
- File root = (File)group.getPath();
- File articleFile = null;
- // this ensures that different threads do not create articles with
- // same number
- while( true ) {
- articleFile = new File(root,(artNum+1)+"");
- if (articleFile.createNewFile()) {
- break;
- }
- }
- getLogger().debug("Copying message to: "+articleFile.getAbsolutePath());
- prop.setProperty(group.getName(),articleFile.getName());
- FileInputStream fin = new FileInputStream(f);
- FileOutputStream fout = new FileOutputStream(articleFile);
- IOUtil.copy(fin,fout);
- fin.close();
- fout.close();
+ FileInputStream newsStream = new FileInputStream(spoolFile);
+ NNTPArticle article = group.addArticle(newsStream);
+ prop.setProperty(group.getName(),article.getArticleNumber() + "");
}
articleIDRepo.addArticle(articleID,prop);
- boolean delSuccess = f.delete();
+ boolean delSuccess = spoolFile.delete();
if ( delSuccess == false ) {
- getLogger().error("Could not delete file: "+f.getAbsolutePath());
+ getLogger().error("Could not delete file: " + spoolFile.getAbsolutePath());
}
}
} // class SpoolerRunnable
1.13 +10 -26 jakarta-james/src/java/org/apache/james/userrepository/AbstractJdbcUsersRepository.java
Index: AbstractJdbcUsersRepository.java
===================================================================
RCS file: /home/cvs/jakarta-james/src/java/org/apache/james/userrepository/AbstractJdbcUsersRepository.java,v
retrieving revision 1.12
retrieving revision 1.13
diff -u -r1.12 -r1.13
--- AbstractJdbcUsersRepository.java 2 Oct 2002 06:12:03 -0000 1.12
+++ AbstractJdbcUsersRepository.java 4 Oct 2002 08:17:49 -0000 1.13
@@ -21,7 +21,7 @@
import org.apache.avalon.framework.context.Context;
import org.apache.avalon.framework.context.ContextException;
import org.apache.avalon.framework.context.Contextualizable;
-import org.apache.james.context.AvalonContextConstants;
+import org.apache.james.context.AvalonContextUtilities;
import org.apache.james.services.User;
import org.apache.james.util.JDBCUtil;
import org.apache.james.util.SqlResources;
@@ -243,30 +243,15 @@
try{
DatabaseMetaData dbMetaData = conn.getMetaData();
- // Initialise the sql strings.
- String fileName = m_sqlFileName.substring("file://".length());
- if (!(fileName.startsWith("/"))) {
- String baseDirectory = "";
- try {
- File applicationHome =
- (File)context.get(AvalonContextConstants.APPLICATION_HOME);
- baseDirectory = applicationHome.toString();
- } catch (ContextException ce) {
- getLogger().fatalError("Encountered exception when resolving application home in Avalon context.", ce);
- throw ce;
- } catch (ClassCastException cce) {
- getLogger().fatalError("Application home object stored in Avalon context was not of type java.io.File.", cce);
- throw cce;
- }
- StringBuffer fileNameBuffer =
- new StringBuffer(128)
- .append(baseDirectory)
- .append(File.separator)
- .append(fileName);
- fileName = fileNameBuffer.toString();
+ File sqlFile = null;
+
+ try {
+ sqlFile = AvalonContextUtilities.getFile(context, m_sqlFileName);
+ } catch (Exception e) {
+ getLogger().fatalError(e.getMessage(), e);
+ throw e;
}
- File sqlFile = (new File(fileName)).getCanonicalFile();
-
+
if (getLogger().isDebugEnabled()) {
logBuffer =
new StringBuffer(256)
@@ -337,8 +322,7 @@
.append(tableName)
.append("\'.");
getLogger().info(logBuffer.toString());
- }
- else {
+ } else {
if (getLogger().isDebugEnabled()) {
getLogger().debug("Using table: " + tableName);
}
1.29 +8 -22 jakarta-james/src/java/org/apache/james/mailrepository/JDBCMailRepository.java
Index: JDBCMailRepository.java
===================================================================
RCS file: /home/cvs/jakarta-james/src/java/org/apache/james/mailrepository/JDBCMailRepository.java,v
retrieving revision 1.28
retrieving revision 1.29
diff -u -r1.28 -r1.29
--- JDBCMailRepository.java 2 Oct 2002 06:12:02 -0000 1.28
+++ JDBCMailRepository.java 4 Oct 2002 08:17:49 -0000 1.29
@@ -24,7 +24,7 @@
import org.apache.avalon.framework.context.ContextException;
import org.apache.avalon.framework.context.Contextualizable;
import org.apache.avalon.framework.logger.AbstractLogEnabled;
-import org.apache.james.context.AvalonContextConstants;
+import org.apache.james.context.AvalonContextUtilities;
import org.apache.james.core.MailImpl;
import org.apache.james.core.MimeMessageWrapper;
import org.apache.james.services.MailRepository;
@@ -309,28 +309,14 @@
try {
// Initialise the sql strings.
- String fileName = sqlFileName.substring("file://".length());
- if (!(fileName.startsWith("/"))) {
- String baseDirectory = "";
- try {
- File applicationHome =
- (File)context.get(AvalonContextConstants.APPLICATION_HOME);
- baseDirectory = applicationHome.toString();
- } catch (ContextException ce) {
- getLogger().fatalError("Encountered exception when resolving application home in Avalon context.", ce);
- throw ce;
- } catch (ClassCastException cce) {
- getLogger().fatalError("Application home object stored in Avalon context was not of type java.io.File.", cce);
- throw cce;
- }
- StringBuffer fileNameBuffer =
- new StringBuffer(128)
- .append(baseDirectory)
- .append(File.separator)
- .append(fileName);
- fileName = fileNameBuffer.toString();
+
+ File sqlFile = null;
+ try {
+ sqlFile = AvalonContextUtilities.getFile(context, sqlFileName);
+ } catch (Exception e) {
+ getLogger().fatalError(e.getMessage(), e);
+ throw e;
}
- File sqlFile = (new File(fileName)).getCanonicalFile();
String resourceName = "org.apache.james.mailrepository.JDBCMailRepository";
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Changes ... who, when, how?
Posted by "Noel J. Bergman" <no...@devtech.com>.
Harmeet,
There has been discussion, but judging from various people's comments, there
is interest in the dicussion.
> Some of my reaction stems from (a) AuthService ...
AuthService is a different issue not related to the this refactoring of
Services and Handlers. We shouldn't transfer concerns between unrelated
areas, unless you see a real connection.
> (b) refactoring commits that seemed to be philosophy driven
More than just philosophy. I don't think that they are simply isomorphic.
In any event, since the code was posted for purposes of review and
discussion prior to being committed, seems to me that we're spending more
time discussing the discussion, whereas you want to be discussing the code
over the weekend. So let's discuss the code. :-)
And I thank you for the compliment.
--- Noel
-----Original Message-----
From: Harmeet Bedi [mailto:harmeet@kodemuse.com]
Sent: Friday, October 11, 2002 13:01
To: James Developers List
Subject: Re: Changes ... who, when, how?
----- Original Message -----
From: "Noel J. Bergman" <no...@devtech.com>
> I agree, but I fail to see the problem. In the case of the watchdog and
> service changes, (a) I believe that there has been prior discussion, (b) I
> believe that after the code was changed based upon prior discussion, it
> was specifically posted for the purpose of inviting further discussion,
> and (c) it hasn't committed before discussion.
I apologize to all esp. Peter if this has been discussed and agreed on
before.
I don't think it is the right fix, but if the list has been through this, no
point in revisiting.
However if this has not been agreed on before I'd prefer if the
Scheduler/Service rearchitecture changes are held off for at least 2-3 days.
I would like to chat a bit more about this. - thanks.
Some of my reaction stems from (a) AuthService(bad bug but that should not
have resulted in Auth and Handler coupling) changes where a discussion was
started and concluded in checkin before any consensus. (b) refactoring
commits that seemed to be philosophy driven - again not bad but there is
very little point in silently changing a part of existing philosophy.
Harmeet
PS: Hey Noel, you seem to be a really good facilitator. :-)
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Changes ... who, when, how?
Posted by Harmeet Bedi <ha...@kodemuse.com>.
----- Original Message -----
From: "Noel J. Bergman" <no...@devtech.com>
> I agree, but I fail to see the problem. In the case of the watchdog and
> service changes, (a) I believe that there has been prior discussion, (b) I
> believe that after the code was changed based upon prior discussion, it
was
> specifically posted for the purpose of inviting further discussion, and
(c)
> it hasn't committed before discussion.
I apologize to all esp. Peter if this has been discussed and agreed on
before.
I don't think it is the right fix, but if the list has been through this, no
point in revisiting.
However if this has not been agreed on before I'd prefer if the
Scheduler/Service rearchitecture changes are held off for at least 2-3 days.
I would like to chat a bit more about this. - thanks.
Some of my reaction stems from (a) AuthService(bad bug but that should not
have resulted in Auth and Handler coupling) changes where a discussion was
started and concluded in checkin before any consensus. (b) refactoring
commits that seemed to be philosophy driven - again not bad but there is
very little point in silently changing a part of existing philosophy.
Harmeet
PS: Hey Noel, you seem to be a really good facilitator. :-)
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Changes ... who, when, how?
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I think the problem is that although we operate in a "commit then review"
fashion
Except for one thing: the code isn't committed! It WAS posted for review.
The message from Peter included the proposed code, a description of what and
why, and then concluded with:
"I've done basic testing on this code, and I am now doing further load
testing. I will continue to do testing BEFORE SUBMISSION [emphasis mine].
[clip]
As always, comments, critiques, and questions are welcome."
As I recall, the changes were made from open discussion over the summer, and
based upon input from several folks, although I guess no one formally called
a vote.
> there should be open discussion of intention and reasoning for significant
re-factorings.
I agree. Seems to me that that's exactly what is (and should be) happening.
At least that's how I interpret his action to post the code for comments,
critiques and criticisms before he commits it.
That is the point of Harmeet's that confuses me. I'm confused about the
issues of why wasn't this discussed before, and why wasn't it discussed
before committing, because it seems to me that neither of those is correct.
The code WAS posted for discussion, and before that, I recall discussions of
watchdog timers and service re-factoring. Andrei submitted the original cut
of service changes months ago. Perhaps it would have been good if Peter had
kept more of a running dialog going over the intervening period, but I
didn't perceive any of the changes to have come out of a vacuum.
> http://jakarta.apache.org/site/source.html
Thanks for that, and the rest of the elaboration. How do we address the
"Status Files"?
> I think Harmeet is implying, and I would agree with,
> is that it shouldn't be beyond the bounds of realism
> to expect a reasoned discussion of the issues *in
> advance* to prevent us from ever getting into the
> position where a change, once commited, is vetoed.
I agree, but I fail to see the problem. In the case of the watchdog and
service changes, (a) I believe that there has been prior discussion, (b) I
believe that after the code was changed based upon prior discussion, it was
specifically posted for the purpose of inviting further discussion, and (c)
it hasn't committed before discussion.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Changes ... who, when, how?
Posted by Danny Angus <da...@apache.org>.
> Of course, each person has his or her own views. Why is this a
> problem, per
> se? Your second sentence is that you don't think the recent changes are
> bad. So then what is the problem you are concerned about?
I think the problem is that although we operate in a "commit then review" fashion, it is naieve to think that this alone is enough, there should be open discussion of intention and reasoning for significant re-factorings. Otherwise, as Harmeet says, we could end up in a loop with competing architectures continually replacing one another. Of course we're not going to be that stupid, but it doesn't mean we shouldn't air our views for discussion first.
> There is always turnover in Open Source projects. Things change. Look at
> the changes from mod_jserv to mod_jk to coyote, or Tomcat 3 to
> Tomcat 4.1 or
> 5.
I bet it was well discussed first, and some sort of consensus reached.
> I don't believe that anyone decides to re-factor / re-architect "just
> because." Generally, I think that people have good reasons and express
> them. I doubt that there will be a loop unless there is conflict amongst
> developers, and I thought that "The Process" covers that issue. No?
If strictly applied it should;
http://jakarta.apache.org/site/source.html
if you read para "Status files" you'll see we haven't been using a status file to track work in progress nad short term plans.
if you also read para "Changes" it makes it quite clear that there should be discussion and consensus on major changes, and any commiter can veto any commited change.
> If a Committer wants to veto a change, they issue a -1.
I think Harmeet is implying, and I would agree with, is that it shouldn't be beyond the bounds of realism to expect a reasoned discussion of the issues *in advance* to prevent us from ever getting into the position where a change, once commited, is vetoed.
d.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Changes ... who, when, how?
Posted by "Noel J. Bergman" <no...@devtech.com>.
Harmeet,
> One of the problems with this checkin is that that each person has his/her
> own views.
> I don't think recent changes are bad. James is now moving better than in
> past, but an overall standard way or an exchange on what should be the
> right way may be better.
Of course, each person has his or her own views. Why is this a problem, per
se? Your second sentence is that you don't think the recent changes are
bad. So then what is the problem you are concerned about?
> If each person decides to refactor/rearchitect on instinct or even in
> response to terrible issues, it would be a bit of loop assuming any
> software would have issues and will(hopefully) outlive it's develepers.
There is always turnover in Open Source projects. Things change. Look at
the changes from mod_jserv to mod_jk to coyote, or Tomcat 3 to Tomcat 4.1 or
5. Look at the major architectural change from Cocoon v1 to Cocoon v2. In
some cases, the architecture was changed by the original author, in other
cases it was changed by someone else. In either case, the architecture
changed in response to needs.
If you have a problem with an architectural change, I think it is absolutely
right and appropriate to raise the issues and for developers to discuss the
merits. When people have had issues, I've seen them discussed here. Some
of have been acted upon, others are pending (for example, a whole bunch of
us want to discuss the next generation of Mailet support, but we've all
agreed to put it off until after the next release).
I don't believe that anyone decides to re-factor / re-architect "just
because." Generally, I think that people have good reasons and express
them. I doubt that there will be a loop unless there is conflict amongst
developers, and I thought that "The Process" covers that issue. No?
With respect to the various recent changes, how do you consider that they
were the work of any one person? A number of people discussed the various
issues over a period of time. Andrei contributed initial changes for
services and connections, Steven Short and Shilpa contributed changes to
LinearProcessor. Various people have been working on IMAP. Peter has
worked on finalizing a number of them to the point where, as a Committer, he
felt that they were in a condition that he'd be willing to support. True,
the commit did often lag behind the discussion by weeks or months, but it
seems to me that generally when the changes were readied, they were first
proposed for review and critique, and then only committed after no
objections.
Are you just raising a general issue of who gets to make changes, when and
how? Is there something different that you'd like to see than the process
being observed? As I understand it, Committers are the arbiters of change.
If a Committer wants to veto a change, they issue a -1. If they don't say
anything, as I understand it from Danny, that indicates passive acceptance.
When changes have been proposed recently, I have only see one -1. It was
from Danny, and he withdrew it after realizing that he'd misread the change.
Can you clarify your concern, and your thoughts about how they can be
addressed?
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: cvs commit: jakarta-james/src/java/org/apache/james/mailrepository JDBCMailRepository.java
Posted by Harmeet Bedi <ha...@kodemuse.com>.
----- Original Message -----
From: <pg...@apache.org>
To: <ja...@apache.org>
Sent: Friday, October 04, 2002 1:17 AM
Subject: cvs commit: jakarta-james/src/java/org/apache/james/mailrepository
JDBCMailRepository.java
> pgoldstein 2002/10/04 01:17:50
>
> Modified: src/java/org/apache/james/util/mordred JdbcDataSource.java
> src/java/org/apache/james/nntpserver NNTPHandler.java
> src/java/org/apache/james/nntpserver/repository
> ArticleIDRepository.java NNTPArticleImpl.java
> NNTPGroup.java NNTPGroupImpl.java
> NNTPRepositoryImpl.java NNTPSpooler.java
> src/java/org/apache/james/userrepository
> AbstractJdbcUsersRepository.java
> src/java/org/apache/james/mailrepository
> JDBCMailRepository.java
> Removed: src/java/org/apache/james/nntpserver LISTGroup.java
> src/java/org/apache/james/nntpserver/repository
> NNTPUtil.java
> Log:
> Rearranged object encapsulation for several classes in the NNTP server
code
One of the problems with this checkin is that that each person has his/her
own views.
I don't think recent changes are bad. James is now moving better than in
past, but an overall standard way or an exchange on what should be the right
way may be better.
If each person decides to refactor/rearchitect on instinct or even in
response to terrible issues, it would be a bit of loop assuming any software
would have issues and will(hopefully) outlive it's develepers.
my 2 cents.
Harmeet
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>