You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by "Noel J. Bergman" <no...@devtech.com> on 2005/10/15 05:56:11 UTC
Working towards a release
So, I'm sitting here starting to work on the release. Amongst my questions
is whether this is 2.3 or 3.0. One basis for the latter are the changes not
so much in API, but in the configuration files. Administrators will need to
make changes, primarily in the smtphandler. But for now, I'm calling it
2.3.
With respect to config.xml:
-- we say that derby is our default, but ALL of the
data sources are commented out, including Derby.
-- the config is putting the database under bin/.
I think we should move that under var/.
As I said, I'm just working my way through things, and posting as I find
something of note. Comments solicited. Help appreciated. :-)
Speaking of which, we need to work on:
-- docs
-- package build. licenses need to be put at the package root
-- web site. Embarrassingly, the AL v1 is showing on the web
site (I noticed that today), even though AL v2 is everywhere
else in JAMES.
-- web site building (just moving things around a bit, to start).
Does anyone have time to help work on these things?
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Working towards a release
Posted by Scott Carr <sc...@progbits.com>.
Noel J. Bergman wrote:
>So, I'm sitting here starting to work on the release. Amongst my questions
>is whether this is 2.3 or 3.0. One basis for the latter are the changes not
>so much in API, but in the configuration files. Administrators will need to
>make changes, primarily in the smtphandler. But for now, I'm calling it
>2.3.
>
>With respect to config.xml:
>
> -- we say that derby is our default, but ALL of the
> data sources are commented out, including Derby.
>
> -- the config is putting the database under bin/.
> I think we should move that under var/.
>
>As I said, I'm just working my way through things, and posting as I find
>something of note. Comments solicited. Help appreciated. :-)
>
>Speaking of which, we need to work on:
>
> -- docs
> -- package build. licenses need to be put at the package root
> -- web site. Embarrassingly, the AL v1 is showing on the web
> site (I noticed that today), even though AL v2 is everywhere
> else in JAMES.
> -- web site building (just moving things around a bit, to start).
>
>Does anyone have time to help work on these things?
>
>
Where would be the best place to help? I know Java pretty good, and can
help in various areas, as needed. What kind of docs are needed?
> --- Noel
>
>
--
Scott Carr
OpenOffice.org
Documentation Maintainer
http://documentation.openoffice.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Working towards a release
Posted by Scott Carr <sc...@progbits.com>.
Noel J. Bergman wrote:
>So, I'm sitting here starting to work on the release. Amongst my questions
>is whether this is 2.3 or 3.0. One basis for the latter are the changes not
>so much in API, but in the configuration files. Administrators will need to
>make changes, primarily in the smtphandler. But for now, I'm calling it
>2.3.
>
>
Configuration changes may warrant a 3.0 release, depending on how much
and how in depth the changes are.
>With respect to config.xml:
>
> -- we say that derby is our default, but ALL of the
> data sources are commented out, including Derby.
>
> -- the config is putting the database under bin/.
> I think we should move that under var/.
>
>
Move under /var definately. /usr is no place for data.
>[snip]
>
>
--
Scott Carr
OpenOffice.org
Documentation Maintainer
http://documentation.openoffice.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: Working towards a release
Posted by "Noel J. Bergman" <no...@devtech.com>.
Serge Knystautas wrote:
> Any chance we can use ASF hardware to conduct these tests
> so other people can help?
Interesting thought. We could ask for a JAMES zone, but we will have to be
very careful (a) not to permit others to access JAMES there for spam, and
(b) to make sure that we don't unduly use resources, e.g., the spinloop
issues I found in postal and rabid.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Working towards a release
Posted by Serge Knystautas <sk...@gmail.com>.
On 10/30/05, Noel J. Bergman <no...@devtech.com> wrote:
> Noel J. Bergman <no...@devtech.com> wrote on Oct 19th:
>
> > I have run a first test, but also encountered a problem. Not with
> > JAMES, but > with the test environment. At home, I run JAMES on
> > one server, postal and rabid on another. I hadn't noticed before,
> > but postal and rabid are not CPU friendly. Basically, they eat the CPU.
>
> I have fixed this, at least sufficiently for more testing. I was also able
> to run a few tests using a second machine. So far so good. One more test,
> and I believe that I will be ready to post a build to be voted on as an
> alpha. The next thing after that will be more aggressive looking for memory
> leaks, and the interesting task of converting my production configuration to
> the new scheme. If that works, great. If not, we have some tweaking to do.
> :-) Plus whatever comes back from user tests.
>
> Sound about right?
Sounds good. Any chance we can use ASF hardware to conduct these
tests so other people can help? (I may have asked this before... my
brain is blanking right now).
--
Serge Knystautas
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Working towards a release
Posted by "Jean T. Anderson" <jt...@apache.org>.
Noel J. Bergman wrote:
> Serge,
>
>
>>Noel J. Bergman wrote
>>
>>>I am still concerned about the Derby related crash that I saw earlier.
>>>Anyone have a clue? And is there anything that we need to do to cleanly
>>>shut Derby down?
>
>
>>I'm at -0 over using Derby at this point.
>
>
> Well, we aren't talking about it being the only database, just the default
> database until a user configures another.
>
>
>>my gut was telling me Derby isn't a production grade DB for the
>>type of heavy IO use that James has.
>
>
> I am leaning towards disagreeing with you. Derby is used extensively and
> reliably in commercial products. I'm not sure what our issue is, so I'd
> suggest that we try to get some help from the Derby folks to evaluate and
> correct our use of it (cc: jta@apache.org).
I'm happy to help. What's the problem? Also --
1) What version of derby is being used? Posting the output from this
command would be the most helpful:
java org.apache.derby.tools.sysinfo
2) Which jdbc driver is being used? (Derby embedded, Derby Network
Client, DB2 UDB Universal JDBC, Object Web C-JDBC, ....)
-jean
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: Working towards a release
Posted by Daniel Perry <d....@netcase.co.uk>.
> AFAIK if it has it's own virtual machine (so it's in network mode) and if
has
> more than 64MB it can gracefully handle such situations. The problem is
that under
> 64MB it can do to much for such situations - it's like an IDE that doesn't
do well
> without a minimal size :).
see below...
> No it's not the same. Derby is not 'full in memory' db, and it has special
> fail safe functions(and recovery too).
> There are however a few tips that must be followed to work efficiently
(and mostly they
> do not apply on any DB - e.g. Mysql)
I thought derby (like hsqldb) could run in either network, or embedded mode.
I think noel is using it in embedded mode: his config suggests that (and
that is the advantage of derby over any other database to james - it's
embedded - doesnt require any external program running).
And in that mode, it shares memory with everything else in the JVM. Fail
safe/recovery functions are great... but cant do anything if they cant
allocate any memory because somthing else in the JVM is hogging it all!
Daniel.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: Working towards a release
Posted by "Noel J. Bergman" <no...@devtech.com>.
Serge,
> Noel J. Bergman wrote
> > I am still concerned about the Derby related crash that I saw earlier.
> > Anyone have a clue? And is there anything that we need to do to cleanly
> > shut Derby down?
> I'm at -0 over using Derby at this point.
Well, we aren't talking about it being the only database, just the default
database until a user configures another.
> my gut was telling me Derby isn't a production grade DB for the
> type of heavy IO use that James has.
I am leaning towards disagreeing with you. Derby is used extensively and
reliably in commercial products. I'm not sure what our issue is, so I'd
suggest that we try to get some help from the Derby folks to evaluate and
correct our use of it (cc: jta@apache.org).
> Maybe configure it as the default database, but still leave
> the file for now? Or dbfile?
I don't believe that we're planning to remove support for file or dbfile any
time soon. I prefer dbfile, actually, for my own use.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Working towards a release
Posted by Serge Knystautas <sk...@gmail.com>.
On 11/7/05, Noel J. Bergman <no...@devtech.com> wrote:
> OK, I ran another set of tests overnight, and didn't see any sign of a
> memory leak.
>
> I am still concerned about the Derby related crash that I saw earlier.
> Anyone have a clue? And is there anything that we need to do to cleanly
> shut Derby down?
I'm at -0 over using Derby at this point. I hate to veto, but my gut
was telling me Derby isn't a production grade DB for the type of heavy
IO use that James has. Maybe configure it as the default database,
but still leave the file for now? Or dbfile?
What do you think we should do at this point re: Derby if we don't
have some easy workaround?
--
Serge Knystautas
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Working towards a release
Posted by Ahmed Mohombe <am...@yahoo.com>.
> Given that the whole crash starts with: java.lang.OutOfMemoryError, then
> maybe that caused the problem with derby.
From the derby list, 64MB for it only looks like not enough.
> Does anyone know anything about derby's behaviour when it runs out of
> memory? I'd imagine that these errors are caught in any code that uses a lot
> of memory (ie loading data) but what about other parts of code that arnt as
> likely to cause that kind of problem?
AFAIK if it has it's own virtual machine (so it's in network mode) and if has
more than 64MB it can gracefully handle such situations. The problem is that under
64MB it can do to much for such situations - it's like an IDE that doesn't do well
without a minimal size :).
> I know very little about derby, but i have used hsqldb before. It had
> similar problems - if one instance of the driver opened a database, other
> instances couldnt write to them (or read iirc).
No it's not the same. Derby is not 'full in memory' db, and it has special
fail safe functions(and recovery too).
There are however a few tips that must be followed to work efficiently (and mostly they
do not apply on any DB - e.g. Mysql)
IMHO the best would be to setup a test system that can be reached by other Apache
members, and than to ask kindly the authors from Derby to have a look(they are also Apache members,
right?).
I think they would be more than happy to help (since this could be a prestige 'powered by' project) :).
Ahmed.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: Working towards a release
Posted by Daniel Perry <d....@netcase.co.uk>.
> I am still concerned about the Derby related crash that I saw earlier.
> Anyone have a clue? And is there anything that we need to do to cleanly
> shut Derby down?
>
> --- Noel
Given that the whole crash starts with: java.lang.OutOfMemoryError, then
maybe that caused the problem with derby.
Does anyone know anything about derby's behaviour when it runs out of
memory? I'd imagine that these errors are caught in any code that uses a lot
of memory (ie loading data) but what about other parts of code that arnt as
likely to cause that kind of problem?
I know very little about derby, but i have used hsqldb before. It had
similar problems - if one instance of the driver opened a database, other
instances couldnt write to them (or read iirc). So if you managed to crash
that instance of the database, you had to restart the container before you
could use it again.
This is a problem with built in java databases, and java in general. It's a
shame there isnt a way to allocate java memory to specific services (ie,
allocate xMB to derby, xMB to core, xMB to remotedelivery, etc).
Have you tried reinstating the memory leak to see if the same crash occurs?
Daniel.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: Working towards a release
Posted by "Noel J. Bergman" <no...@devtech.com>.
> [...] The next
> thing after that will be more aggressive looking for memory
> leaks, and the interesting task of converting my production
> configuration to the new scheme. If that works, great. If
> not, we have some tweaking to do.
> :-) Plus whatever comes back from user tests.
OK, I ran another set of tests overnight, and didn't see any sign of a
memory leak.
I am still concerned about the Derby related crash that I saw earlier.
Anyone have a clue? And is there anything that we need to do to cleanly
shut Derby down?
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Working towards a release
Posted by Stefano Bagnara <ap...@bago.org>.
> [...] The next
> thing after that will be more aggressive looking for memory
> leaks, and the interesting task of converting my production
> configuration to the new scheme. If that works, great. If
> not, we have some tweaking to do.
> :-) Plus whatever comes back from user tests.
>
> Sound about right?
>
> --- Noel
+1
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: Working towards a release
Posted by "Noel J. Bergman" <no...@devtech.com>.
Noel J. Bergman <no...@devtech.com> wrote on Oct 19th:
> I have run a first test, but also encountered a problem. Not with
> JAMES, but > with the test environment. At home, I run JAMES on
> one server, postal and rabid on another. I hadn't noticed before,
> but postal and rabid are not CPU friendly. Basically, they eat the CPU.
I have fixed this, at least sufficiently for more testing. I was also able
to run a few tests using a second machine. So far so good. One more test,
and I believe that I will be ready to post a build to be voted on as an
alpha. The next thing after that will be more aggressive looking for memory
leaks, and the interesting task of converting my production configuration to
the new scheme. If that works, great. If not, we have some tweaking to do.
:-) Plus whatever comes back from user tests.
Sound about right?
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: Working towards a release
Posted by "Noel J. Bergman" <no...@devtech.com>.
Noel J. Bergman <no...@devtech.com> wrote:
> Planning on it. I need to re-do my testing configuration with the new
> config and test that, and then re-do my production configuration and give
> that a whirl. If those look good, I'll propose an alpha for vote. I
don't
> like posting anything that I would not run myself.
I have run a first test, but also encountered a problem. Not with JAMES,
but with the test environment. At home, I run JAMES on one server, postal
and rabid on another. I hadn't noticed before, but postal and rabid are not
CPU friendly. Basically, they eat the CPU. So running postal and rabid
under VMware on my laptop along with JAMES means that they interfer with
testing, so I had to back off the testing a bit. Just rebuilt postal and
rabid for Mac OS X, so we'll run those on a separate physical machine. And
I have notified Russell of the issue.
I have run an overnight test with just SMTP and SMTP relay, using db for
mailboxes and users, and file for all else. No perceived memory leaks, but
I did run out of handles for the database until I raised the max to 40.
Bottom line: so far, so good. Will start additional testing along the full
SMTP and POP3 path tonight, and will check file, db and dbfile using Derby
for the DB. Do not plan to test any other databases.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Working towards a release
Posted by Serge Knystautas <sk...@gmail.com>.
On 10/17/05, Noel J. Bergman <no...@devtech.com> wrote:
> Planning on it. I need to re-do my testing configuration with the new
> config and test that, and then re-do my production configuration and give
> that a whirl. If those look good, I'll propose an alpha for vote. I don't
> like posting anything that I would not run myself.
Sounds great. Thanks!
--
Serge Knystautas
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: Working towards a release
Posted by "Noel J. Bergman" <no...@devtech.com>.
Serge Knystautas wrote:
> This feels extremely lame to say, but why don't we just put an alpha
> release out (once you get derby properly set as default, steps to
> migrate, whatever core issues you addressed) and not worry about all
> the other project-level issues?
Planning on it. I need to re-do my testing configuration with the new
config and test that, and then re-do my production configuration and give
that a whirl. If those look good, I'll propose an alpha for vote. I don't
like posting anything that I would not run myself.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Working towards a release
Posted by Serge Knystautas <sk...@gmail.com>.
On 10/15/05, Scott Carr <sc...@progbits.com> wrote:
> >make changes, primarily in the smtphandler. But for now, I'm calling it
> >2.3.
> >
> Configuration changes may warrant a 3.0 release, depending on how much
> and how in depth the changes are.
+1
This feels extremely lame to say, but why don't we just put an alpha
release out (once you get derby properly set as default, steps to
migrate, whatever core issues you addressed) and not worry about all
the other project-level issues?
No documentation is bad, but IMO no releases is worse. :(
--
Serge Knystautas
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Working towards a release
Posted by Stefano Bagnara <ap...@bago.org>.
> So, I'm sitting here starting to work on the release.
> Amongst my questions is whether this is 2.3 or 3.0. One
> basis for the latter are the changes not so much in API, but
> in the configuration files. Administrators will need to make
> changes, primarily in the smtphandler. But for now, I'm
> calling it 2.3.
+1.
> With respect to config.xml:
>
> -- we say that derby is our default, but ALL of the
> data sources are commented out, including Derby.
I'm +1 for this change.
http://issues.apache.org/jira/browse/JAMES-393?page=all
> -- the config is putting the database under bin/.
> I think we should move that under var/.
Thank you for the fix.
> As I said, I'm just working my way through things, and
> posting as I find something of note. Comments solicited.
> Help appreciated. :-)
>
> Speaking of which, we need to work on:
>
> -- docs
> -- package build. licenses need to be put at the package root
> -- web site. Embarrassingly, the AL v1 is showing on the web
> site (I noticed that today), even though AL v2 is everywhere
> else in JAMES.
> -- web site building (just moving things around a bit, to start).
>
> Does anyone have time to help work on these things?
>
> --- Noel
I currently don't have too much time. Maybe I should try to keep the time
for code issues.
I don't know much about documentation generation, website rules and release
packaging.
I don't know wether this is a possible roadmap or not but I think we could
publish a first 2.3.0 alpha 1 release and from there decide what
changes/bugfix we need for the following release and delay the goal to
update website/docs for the 2.3.0rc1.
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org