You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by Karl Wright <da...@gmail.com> on 2012/09/21 02:46:06 UTC
[VOTE] Release Apache ManifoldCF 1.0, RC0
Please vote +1 to release ManifoldCF 1.0, RC0. The release artifact
can be found at:
http://people.apache.org/~kwright/apache-manifoldcf-1.0
There is also an SVN tag at:
https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.0-RC0
Karl
[WITHDRAW][VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Karl Wright <da...@gmail.com>.
Withdrawn; spinning RC1 now.
Karl
On Fri, Sep 21, 2012 at 2:55 PM, Karl Wright <da...@gmail.com> wrote:
> Hi,
>
> I see what has happened here. You unregistered the connectors before
> you deleted the job. That basically meant that the job cleanup can't
> take place until the connector(s) it requires are registered again.
>
> This is "as designed". You should be aware that it is unnecessary to
> delete jobs and connections when you upgrade. Nor is
> unregistering/reregistering your connectors.
>
> I'm going to spin a new RC with the NPE fix and a bunch of other
> things - but I'm pretty certain the mystery is solved.
>
> Thanks,
> Karl
>
> On Fri, Sep 21, 2012 at 10:17 AM, Karl Wright <da...@gmail.com> wrote:
>> Thanks - have a good weekend.
>>
>> I will also withhold my vote for this RC until this is ironed out.
>> What I'm worried about is that all of the startup/shutdown model
>> changes might have impacted ManifoldCF running on Resin, so I think it
>> is essential we wait until this is resolved before declaring any
>> release candidate "done".
>>
>> Karl
>>
>>
>> On Fri, Sep 21, 2012 at 10:12 AM, Erlend Garåsen
>> <e....@usit.uio.no> wrote:
>>>
>>> I have created a ticket and attached _simplereport__jsp.java.
>>>
>>> I will catch a train early tomorrow morning up to the mountains and will
>>> therefore be unavailable for a couple of days. I can investigate this
>>> further on Monday.
>>>
>>> Happy weekend to you all!
>>>
>>>
>>> Erlend
>>> --
>>> Erlend Garåsen
>>> Center for Information Technology Services
>>> University of Oslo
>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Erlend Garåsen <e....@usit.uio.no>.
Interesting.
Anyway, it seems that things are going a lot more better with RC3 on
Resin. I will test it more properly this week and vote as soon as possible.
Erlend
On 25.09.12 01.48, Karl Wright wrote:
> I checked things out and did find a problem - see CONNECTORS-539. I
> resolved that issue, and tried out the scenario I think you were
> running into, and everything worked just as expected.
>
> I've spun another RC now (RC2). Here's hoping that it will work as advertised.
>
> Karl
>
>
> On Mon, Sep 24, 2012 at 10:09 AM, Karl Wright <da...@gmail.com> wrote:
>> Hi, please see below:
>>
>> On Mon, Sep 24, 2012 at 9:59 AM, Erlend Garåsen <e....@usit.uio.no> wrote:
>>> On 21.09.12 20.55, Karl Wright wrote:
>>>
>>>> I see what has happened here. You unregistered the connectors before
>>>> you deleted the job. That basically meant that the job cleanup can't
>>>> take place until the connector(s) it requires are registered again.
>>>
>>>
>>> That's correct, and I also see the problem now. I forgot to install the
>>> Filesystem connector before I did the configuration import (normally we do
>>> not use this connector, but I installed one as a part of a test I did).
>>>
>>> After I installed it, I do not longer get an NPE.
>>>
>>
>> ok, that is good.
>>
>>> Maybe our routines for upgrading MCF need to be changed. We want to be sure
>>> that these connectors do not need new fields in the database tables due to
>>> changed/new functions.
>>
>> This should happen automatically if you simply reregister the
>> connector. You do not need to unregister it, I am pretty sure.
>>
>> Therefore I thought this was the safest approach.
>>> First we export the configuration, then we uninstall all connectors by using
>>> the executecommand script, then deleting the tables by performing an
>>> "agents.Uninstall" command, then reinstall everything and finally import the
>>> configuration.
>>>
>>
>> You definitely do not need to do all that. Just re-register the
>> crawlerAgent and the connectors. It should automatically upgrade
>> database tables when you do that.
>>
>>> Still I cannot delete my jobs since their statuses are "cleaning up". And
>>> the reason is because I didn't delete my jobs prior to executing
>>> "crawler.UnRegisterAll"?
>>>
>>
>> Once you register the missing connectors again, and have the agents
>> process running, the job should delete. If this is not happening it
>> may just take a few minutes for things to wake up, but it should pick
>> up where it left off.
>>
>> If it doesn't seem to recover, it may be time to look in the database
>> to see what the status is of all the connectors. When you unregister
>> and reregister a connector, it goes through the jobs that depend on
>> the connector and changes their state. If there is something wrong
>> with that logic then the connector will be registered but the job will
>> still be in the "unregistered" state.
>>
>> I'll try to reproduce this situation here sometime this evening, to be
>> sure that everything recovers the way it is supposed to.
>>
>> Karl
>>
>>>
>>> Erlend
>>>
>>> --
>>> Erlend Garåsen
>>> Center for Information Technology Services
>>> University of Oslo
>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
--
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Karl Wright <da...@gmail.com>.
I checked things out and did find a problem - see CONNECTORS-539. I
resolved that issue, and tried out the scenario I think you were
running into, and everything worked just as expected.
I've spun another RC now (RC2). Here's hoping that it will work as advertised.
Karl
On Mon, Sep 24, 2012 at 10:09 AM, Karl Wright <da...@gmail.com> wrote:
> Hi, please see below:
>
> On Mon, Sep 24, 2012 at 9:59 AM, Erlend Garåsen <e....@usit.uio.no> wrote:
>> On 21.09.12 20.55, Karl Wright wrote:
>>
>>> I see what has happened here. You unregistered the connectors before
>>> you deleted the job. That basically meant that the job cleanup can't
>>> take place until the connector(s) it requires are registered again.
>>
>>
>> That's correct, and I also see the problem now. I forgot to install the
>> Filesystem connector before I did the configuration import (normally we do
>> not use this connector, but I installed one as a part of a test I did).
>>
>> After I installed it, I do not longer get an NPE.
>>
>
> ok, that is good.
>
>> Maybe our routines for upgrading MCF need to be changed. We want to be sure
>> that these connectors do not need new fields in the database tables due to
>> changed/new functions.
>
> This should happen automatically if you simply reregister the
> connector. You do not need to unregister it, I am pretty sure.
>
> Therefore I thought this was the safest approach.
>> First we export the configuration, then we uninstall all connectors by using
>> the executecommand script, then deleting the tables by performing an
>> "agents.Uninstall" command, then reinstall everything and finally import the
>> configuration.
>>
>
> You definitely do not need to do all that. Just re-register the
> crawlerAgent and the connectors. It should automatically upgrade
> database tables when you do that.
>
>> Still I cannot delete my jobs since their statuses are "cleaning up". And
>> the reason is because I didn't delete my jobs prior to executing
>> "crawler.UnRegisterAll"?
>>
>
> Once you register the missing connectors again, and have the agents
> process running, the job should delete. If this is not happening it
> may just take a few minutes for things to wake up, but it should pick
> up where it left off.
>
> If it doesn't seem to recover, it may be time to look in the database
> to see what the status is of all the connectors. When you unregister
> and reregister a connector, it goes through the jobs that depend on
> the connector and changes their state. If there is something wrong
> with that logic then the connector will be registered but the job will
> still be in the "unregistered" state.
>
> I'll try to reproduce this situation here sometime this evening, to be
> sure that everything recovers the way it is supposed to.
>
> Karl
>
>>
>> Erlend
>>
>> --
>> Erlend Garåsen
>> Center for Information Technology Services
>> University of Oslo
>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
RE: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by ka...@nokia.com.
Yes, this is all unneeded and superfluous. Worse, it loses state between upgrades.
Karl
-----Original Message-----
From: ext Erlend Garåsen [mailto:e.f.garasen@usit.uio.no]
Sent: Monday, September 24, 2012 10:51 AM
To: dev@manifoldcf.apache.org
Subject: Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
On 24.09.12 16.45, Erlend Garåsen wrote:
>> This should happen automatically if you simply reregister the
>> connector. You do not need to unregister it, I am pretty sure.
>
> Thanks. This makes our upgrading routines far more easier. I will
> update our routine document right away.
Then I guess the following is also unnecessary to do prior to an upgrade?
$MCF_HOME/processes/script/executecommand.sh
org.apache.manifoldcf.agents.UnRegisterAllOutputs
$MCF_HOME/processes/script/executecommand.sh
org.apache.manifoldcf.agents.UnRegisterAll
$MCF_HOME/processes/script/executecommand.sh
org.apache.manifoldcf.agents.Uninstall
Erlend
--
Erlend Garåsen
Center for Information Technology Services University of Oslo P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Erlend Garåsen <e....@usit.uio.no>.
On 24.09.12 16.45, Erlend Garåsen wrote:
>> This should happen automatically if you simply reregister the
>> connector. You do not need to unregister it, I am pretty sure.
>
> Thanks. This makes our upgrading routines far more easier. I will update
> our routine document right away.
Then I guess the following is also unnecessary to do prior to an upgrade?
$MCF_HOME/processes/script/executecommand.sh
org.apache.manifoldcf.agents.UnRegisterAllOutputs
$MCF_HOME/processes/script/executecommand.sh
org.apache.manifoldcf.agents.UnRegisterAll
$MCF_HOME/processes/script/executecommand.sh
org.apache.manifoldcf.agents.Uninstall
Erlend
--
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Erlend Garåsen <e....@usit.uio.no>.
On 24.09.12 16.09, Karl Wright wrote:
>> Maybe our routines for upgrading MCF need to be changed. We want to be sure
>> that these connectors do not need new fields in the database tables due to
>> changed/new functions.
>
> This should happen automatically if you simply reregister the
> connector. You do not need to unregister it, I am pretty sure.
Thanks. This makes our upgrading routines far more easier. I will update
our routine document right away.
>> Still I cannot delete my jobs since their statuses are "cleaning up". And
>> the reason is because I didn't delete my jobs prior to executing
>> "crawler.UnRegisterAll"?
>>
>
> Once you register the missing connectors again, and have the agents
> process running, the job should delete. If this is not happening it
> may just take a few minutes for things to wake up, but it should pick
> up where it left off.
>
> If it doesn't seem to recover, it may be time to look in the database
> to see what the status is of all the connectors. When you unregister
> and reregister a connector, it goes through the jobs that depend on
> the connector and changes their state. If there is something wrong
> with that logic then the connector will be registered but the job will
> still be in the "unregistered" state.
Great. The Agent process is always running on our test server with a
cron job checking its status every day. I can check again tomorrow in
case the jobs have been deleted.
I will install RC1 and vote later this week.
Erlend
--
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Karl Wright <da...@gmail.com>.
Hi, please see below:
On Mon, Sep 24, 2012 at 9:59 AM, Erlend Garåsen <e....@usit.uio.no> wrote:
> On 21.09.12 20.55, Karl Wright wrote:
>
>> I see what has happened here. You unregistered the connectors before
>> you deleted the job. That basically meant that the job cleanup can't
>> take place until the connector(s) it requires are registered again.
>
>
> That's correct, and I also see the problem now. I forgot to install the
> Filesystem connector before I did the configuration import (normally we do
> not use this connector, but I installed one as a part of a test I did).
>
> After I installed it, I do not longer get an NPE.
>
ok, that is good.
> Maybe our routines for upgrading MCF need to be changed. We want to be sure
> that these connectors do not need new fields in the database tables due to
> changed/new functions.
This should happen automatically if you simply reregister the
connector. You do not need to unregister it, I am pretty sure.
Therefore I thought this was the safest approach.
> First we export the configuration, then we uninstall all connectors by using
> the executecommand script, then deleting the tables by performing an
> "agents.Uninstall" command, then reinstall everything and finally import the
> configuration.
>
You definitely do not need to do all that. Just re-register the
crawlerAgent and the connectors. It should automatically upgrade
database tables when you do that.
> Still I cannot delete my jobs since their statuses are "cleaning up". And
> the reason is because I didn't delete my jobs prior to executing
> "crawler.UnRegisterAll"?
>
Once you register the missing connectors again, and have the agents
process running, the job should delete. If this is not happening it
may just take a few minutes for things to wake up, but it should pick
up where it left off.
If it doesn't seem to recover, it may be time to look in the database
to see what the status is of all the connectors. When you unregister
and reregister a connector, it goes through the jobs that depend on
the connector and changes their state. If there is something wrong
with that logic then the connector will be registered but the job will
still be in the "unregistered" state.
I'll try to reproduce this situation here sometime this evening, to be
sure that everything recovers the way it is supposed to.
Karl
>
> Erlend
>
> --
> Erlend Garåsen
> Center for Information Technology Services
> University of Oslo
> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Erlend Garåsen <e....@usit.uio.no>.
On 21.09.12 20.55, Karl Wright wrote:
> I see what has happened here. You unregistered the connectors before
> you deleted the job. That basically meant that the job cleanup can't
> take place until the connector(s) it requires are registered again.
That's correct, and I also see the problem now. I forgot to install the
Filesystem connector before I did the configuration import (normally we
do not use this connector, but I installed one as a part of a test I did).
After I installed it, I do not longer get an NPE.
Maybe our routines for upgrading MCF need to be changed. We want to be
sure that these connectors do not need new fields in the database tables
due to changed/new functions. Therefore I thought this was the safest
approach. First we export the configuration, then we uninstall all
connectors by using the executecommand script, then deleting the tables
by performing an "agents.Uninstall" command, then reinstall everything
and finally import the configuration.
Still I cannot delete my jobs since their statuses are "cleaning up".
And the reason is because I didn't delete my jobs prior to executing
"crawler.UnRegisterAll"?
Erlend
--
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Karl Wright <da...@gmail.com>.
Hi,
I see what has happened here. You unregistered the connectors before
you deleted the job. That basically meant that the job cleanup can't
take place until the connector(s) it requires are registered again.
This is "as designed". You should be aware that it is unnecessary to
delete jobs and connections when you upgrade. Nor is
unregistering/reregistering your connectors.
I'm going to spin a new RC with the NPE fix and a bunch of other
things - but I'm pretty certain the mystery is solved.
Thanks,
Karl
On Fri, Sep 21, 2012 at 10:17 AM, Karl Wright <da...@gmail.com> wrote:
> Thanks - have a good weekend.
>
> I will also withhold my vote for this RC until this is ironed out.
> What I'm worried about is that all of the startup/shutdown model
> changes might have impacted ManifoldCF running on Resin, so I think it
> is essential we wait until this is resolved before declaring any
> release candidate "done".
>
> Karl
>
>
> On Fri, Sep 21, 2012 at 10:12 AM, Erlend Garåsen
> <e....@usit.uio.no> wrote:
>>
>> I have created a ticket and attached _simplereport__jsp.java.
>>
>> I will catch a train early tomorrow morning up to the mountains and will
>> therefore be unavailable for a couple of days. I can investigate this
>> further on Monday.
>>
>> Happy weekend to you all!
>>
>>
>> Erlend
>> --
>> Erlend Garåsen
>> Center for Information Technology Services
>> University of Oslo
>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Karl Wright <da...@gmail.com>.
Thanks - have a good weekend.
I will also withhold my vote for this RC until this is ironed out.
What I'm worried about is that all of the startup/shutdown model
changes might have impacted ManifoldCF running on Resin, so I think it
is essential we wait until this is resolved before declaring any
release candidate "done".
Karl
On Fri, Sep 21, 2012 at 10:12 AM, Erlend Garåsen
<e....@usit.uio.no> wrote:
>
> I have created a ticket and attached _simplereport__jsp.java.
>
> I will catch a train early tomorrow morning up to the mountains and will
> therefore be unavailable for a couple of days. I can investigate this
> further on Monday.
>
> Happy weekend to you all!
>
>
> Erlend
> --
> Erlend Garåsen
> Center for Information Technology Services
> University of Oslo
> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Erlend Garåsen <e....@usit.uio.no>.
I have created a ticket and attached _simplereport__jsp.java.
I will catch a train early tomorrow morning up to the mountains and will
therefore be unavailable for a couple of days. I can investigate this
further on Monday.
Happy weekend to you all!
Erlend
--
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Erlend Garåsen <e....@usit.uio.no>.
According to my bash history, I did a LockClean prior to the upgrade:
solr-test02 mcf-1 $ history | grep LockClean
154 $MCF_HOME/processes/script/executecommand.sh
org.apache.manifoldcf.core.LockClean
I will create a ticket. Let's hope this is a local problem on my server
and not a bug, but I think it's best to investigate it to be sure.
I will try to provide as much information as possible in the ticket.
Erlend
On 21.09.12 15.08, Karl Wright wrote:
> Hmm, that's not good.
>
> Can you open a ticket for this NPE, and also please attach the .java
> file it is referring to: _simplereport__jsp.java? It should be found
> in resin's workarea somewhere.
>
> As for the job not being deleted, can you supply further details of
> your setup? Specifically, properties.xml (so I can see what synch
> settings you have), and what database, etc.
>
> If there is nothing in the log, I'd shut down everything, execute the
> lock-clean procedure, and start everything back up, and see if that
> fixed the issue.
>
> Karl
>
>
> On Fri, Sep 21, 2012 at 9:01 AM, Erlend Garåsen <e....@usit.uio.no> wrote:
>> On 21.09.12 14.47, Karl Wright wrote:
>>>
>>> A temporary error should not block a (non running) job from getting
>>> cleaned up. The job can't be deleted until it is stopped, and no
>>> outstanding documents are being worked on.
>>>
>>> How many documents are still listed for the job in the UI?
>>
>>
>> Documents: 0
>> Active: 0
>> Proceed: 0
>>
>> Nothing in the simple history, but I get a 500 Server Exceptions if I try to
>> read the history of the file connector (due to a NPE) - stack trace further
>> down in this post.
>>
>> I can try to unregister all connectors and empty the database and try again.
>>
>> [2012-09-21 14:57:10.220] {resin-port-127.0.0.1:6945-143}
>> java.lang.NullPointerException
>> at
>> _jsp._simplereport__jsp._jspService(_simplereport__jsp.java:367)
>> at
>> _jsp._simplereport__jsp._jspService(_simplereport__jsp.java:36)
>> at
>> com.caucho.jsp.JavaPage.service(JavaPage.java:64)
>> at
>> com.caucho.jsp.Page.pageservice(Page.java:542)
>> at
>> com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:194)
>> at
>> com.caucho.server.webapp.DispatchFilterChain.doFilter(DispatchFilterChain.java:126)
>> at
>> com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:289)
>> at
>> com.caucho.server.webapp.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:298)
>> at
>> com.caucho.server.webapp.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:116)
>> at
>> com.caucho.jsp.PageContextImpl.forward(PageContextImpl.java:1149)
>> at
>> _jsp._execute__jsp._jspService(_execute__jsp.java:1072)
>> at
>> _jsp._execute__jsp._jspService(_execute__jsp.java:36)
>> at
>> com.caucho.jsp.JavaPage.service(JavaPage.java:64)
>> at
>> com.caucho.jsp.Page.pageservice(Page.java:542)
>> at
>> com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:194)
>> at
>> com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:156)
>> at
>> com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:289)
>> at
>> com.caucho.server.hmux.HmuxRequest.handleInvocation(HmuxRequest.java:468)
>> at
>> com.caucho.server.hmux.HmuxRequest.handleRequestImpl(HmuxRequest.java:369)
>> at
>> com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:336)
>> at
>> com.caucho.network.listen.TcpSocketLink.dispatchRequest(TcpSocketLink.java:1301)
>> at
>> com.caucho.network.listen.TcpSocketLink.handleRequest(TcpSocketLink.java:1257)
>> at
>> com.caucho.network.listen.TcpSocketLink.handleRequestsImpl(TcpSocketLink.java:1241)
>> at
>> com.caucho.network.listen.TcpSocketLink.handleRequests(TcpSocketLink.java:1149)
>> at
>> com.caucho.network.listen.TcpSocketLink.handleAcceptTaskImpl(TcpSocketLink.java:950)
>> at
>> com.caucho.network.listen.TcpSocketLink.handleAcceptTask(TcpSocketLink.java:902)
>> at
>> com.caucho.network.listen.ConnectionTask.runThread(ConnectionTask.java:114)
>> at
>> com.caucho.network.listen.ConnectionTask.run(ConnectionTask.java:93)
>> at
>> com.caucho.network.listen.SocketLinkThreadLauncher.handleTasks(SocketLinkThreadLauncher.java:169)
>> at
>> com.caucho.network.listen.TcpSocketResumeThread.run(TcpSocketResumeThread.java:62)
>> at
>> com.caucho.env.thread2.ResinThread2.runTasks(ResinThread2.java:173)
>> at
>> com.caucho.env.thread2.ResinThread2.run(ResinThread2.java:118)
>>
>>
>>
>>
>>>
>>> Karl
>>>
>>> On Fri, Sep 21, 2012 at 8:36 AM, Erlend Garåsen <e....@usit.uio.no>
>>> wrote:
>>>>
>>>> On 21.09.12 14.07, Karl Wright wrote:
>>>>
>>>>> If you think the agents process is running, and yet you cannot delete
>>>>> a job, you should see stuff in the manifoldcf log that would indicate
>>>>> what the trouble likely is.
>>>>
>>>>
>>>>
>>>> solr-test02 mcf-1 $ /www/var/data/mcf/mcf-1/agentctl start
>>>> Error: MCF Agent seems to be running already with PID 26310
>>>>
>>>> You're right. It has some problems with several documents:
>>>> WARN 2012-08-10 14:25:10,421 (Worker thread '4') - Pre-ingest service
>>>> interruption reported for job 1335434066944 connection 'web_crawler':
>>>> Http
>>>> response temporary error on
>>>> 'http://www.apollon.uio.no/vis/art/1998/3/intervju': 503
>>>>
>>>> So I will ignore this at the moment and try to test a little bit more
>>>> with
>>>> other parts of MCF.
>>>>
>>>> Erlend
>>>>
>>>>
>>>> --
>>>> Erlend Garåsen
>>>> Center for Information Technology Services
>>>> University of Oslo
>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP:
>>>> 31050
>>
>>
>>
>> --
>> Erlend Garåsen
>> Center for Information Technology Services
>> University of Oslo
>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
--
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Karl Wright <da...@gmail.com>.
Hmm, that's not good.
Can you open a ticket for this NPE, and also please attach the .java
file it is referring to: _simplereport__jsp.java? It should be found
in resin's workarea somewhere.
As for the job not being deleted, can you supply further details of
your setup? Specifically, properties.xml (so I can see what synch
settings you have), and what database, etc.
If there is nothing in the log, I'd shut down everything, execute the
lock-clean procedure, and start everything back up, and see if that
fixed the issue.
Karl
On Fri, Sep 21, 2012 at 9:01 AM, Erlend Garåsen <e....@usit.uio.no> wrote:
> On 21.09.12 14.47, Karl Wright wrote:
>>
>> A temporary error should not block a (non running) job from getting
>> cleaned up. The job can't be deleted until it is stopped, and no
>> outstanding documents are being worked on.
>>
>> How many documents are still listed for the job in the UI?
>
>
> Documents: 0
> Active: 0
> Proceed: 0
>
> Nothing in the simple history, but I get a 500 Server Exceptions if I try to
> read the history of the file connector (due to a NPE) - stack trace further
> down in this post.
>
> I can try to unregister all connectors and empty the database and try again.
>
> [2012-09-21 14:57:10.220] {resin-port-127.0.0.1:6945-143}
> java.lang.NullPointerException
> at
> _jsp._simplereport__jsp._jspService(_simplereport__jsp.java:367)
> at
> _jsp._simplereport__jsp._jspService(_simplereport__jsp.java:36)
> at
> com.caucho.jsp.JavaPage.service(JavaPage.java:64)
> at
> com.caucho.jsp.Page.pageservice(Page.java:542)
> at
> com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:194)
> at
> com.caucho.server.webapp.DispatchFilterChain.doFilter(DispatchFilterChain.java:126)
> at
> com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:289)
> at
> com.caucho.server.webapp.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:298)
> at
> com.caucho.server.webapp.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:116)
> at
> com.caucho.jsp.PageContextImpl.forward(PageContextImpl.java:1149)
> at
> _jsp._execute__jsp._jspService(_execute__jsp.java:1072)
> at
> _jsp._execute__jsp._jspService(_execute__jsp.java:36)
> at
> com.caucho.jsp.JavaPage.service(JavaPage.java:64)
> at
> com.caucho.jsp.Page.pageservice(Page.java:542)
> at
> com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:194)
> at
> com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:156)
> at
> com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:289)
> at
> com.caucho.server.hmux.HmuxRequest.handleInvocation(HmuxRequest.java:468)
> at
> com.caucho.server.hmux.HmuxRequest.handleRequestImpl(HmuxRequest.java:369)
> at
> com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:336)
> at
> com.caucho.network.listen.TcpSocketLink.dispatchRequest(TcpSocketLink.java:1301)
> at
> com.caucho.network.listen.TcpSocketLink.handleRequest(TcpSocketLink.java:1257)
> at
> com.caucho.network.listen.TcpSocketLink.handleRequestsImpl(TcpSocketLink.java:1241)
> at
> com.caucho.network.listen.TcpSocketLink.handleRequests(TcpSocketLink.java:1149)
> at
> com.caucho.network.listen.TcpSocketLink.handleAcceptTaskImpl(TcpSocketLink.java:950)
> at
> com.caucho.network.listen.TcpSocketLink.handleAcceptTask(TcpSocketLink.java:902)
> at
> com.caucho.network.listen.ConnectionTask.runThread(ConnectionTask.java:114)
> at
> com.caucho.network.listen.ConnectionTask.run(ConnectionTask.java:93)
> at
> com.caucho.network.listen.SocketLinkThreadLauncher.handleTasks(SocketLinkThreadLauncher.java:169)
> at
> com.caucho.network.listen.TcpSocketResumeThread.run(TcpSocketResumeThread.java:62)
> at
> com.caucho.env.thread2.ResinThread2.runTasks(ResinThread2.java:173)
> at
> com.caucho.env.thread2.ResinThread2.run(ResinThread2.java:118)
>
>
>
>
>>
>> Karl
>>
>> On Fri, Sep 21, 2012 at 8:36 AM, Erlend Garåsen <e....@usit.uio.no>
>> wrote:
>>>
>>> On 21.09.12 14.07, Karl Wright wrote:
>>>
>>>> If you think the agents process is running, and yet you cannot delete
>>>> a job, you should see stuff in the manifoldcf log that would indicate
>>>> what the trouble likely is.
>>>
>>>
>>>
>>> solr-test02 mcf-1 $ /www/var/data/mcf/mcf-1/agentctl start
>>> Error: MCF Agent seems to be running already with PID 26310
>>>
>>> You're right. It has some problems with several documents:
>>> WARN 2012-08-10 14:25:10,421 (Worker thread '4') - Pre-ingest service
>>> interruption reported for job 1335434066944 connection 'web_crawler':
>>> Http
>>> response temporary error on
>>> 'http://www.apollon.uio.no/vis/art/1998/3/intervju': 503
>>>
>>> So I will ignore this at the moment and try to test a little bit more
>>> with
>>> other parts of MCF.
>>>
>>> Erlend
>>>
>>>
>>> --
>>> Erlend Garåsen
>>> Center for Information Technology Services
>>> University of Oslo
>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP:
>>> 31050
>
>
>
> --
> Erlend Garåsen
> Center for Information Technology Services
> University of Oslo
> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Erlend Garåsen <e....@usit.uio.no>.
On 21.09.12 14.47, Karl Wright wrote:
> A temporary error should not block a (non running) job from getting
> cleaned up. The job can't be deleted until it is stopped, and no
> outstanding documents are being worked on.
>
> How many documents are still listed for the job in the UI?
Documents: 0
Active: 0
Proceed: 0
Nothing in the simple history, but I get a 500 Server Exceptions if I
try to read the history of the file connector (due to a NPE) - stack
trace further down in this post.
I can try to unregister all connectors and empty the database and try again.
[2012-09-21 14:57:10.220] {resin-port-127.0.0.1:6945-143}
java.lang.NullPointerException
at
_jsp._simplereport__jsp._jspService(_simplereport__jsp.java:367)
at
_jsp._simplereport__jsp._jspService(_simplereport__jsp.java:36)
at
com.caucho.jsp.JavaPage.service(JavaPage.java:64)
at
com.caucho.jsp.Page.pageservice(Page.java:542)
at
com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:194)
at
com.caucho.server.webapp.DispatchFilterChain.doFilter(DispatchFilterChain.java:126)
at
com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:289)
at
com.caucho.server.webapp.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:298)
at
com.caucho.server.webapp.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:116)
at
com.caucho.jsp.PageContextImpl.forward(PageContextImpl.java:1149)
at
_jsp._execute__jsp._jspService(_execute__jsp.java:1072)
at
_jsp._execute__jsp._jspService(_execute__jsp.java:36)
at
com.caucho.jsp.JavaPage.service(JavaPage.java:64)
at
com.caucho.jsp.Page.pageservice(Page.java:542)
at
com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:194)
at
com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:156)
at
com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:289)
at
com.caucho.server.hmux.HmuxRequest.handleInvocation(HmuxRequest.java:468)
at
com.caucho.server.hmux.HmuxRequest.handleRequestImpl(HmuxRequest.java:369)
at
com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:336)
at
com.caucho.network.listen.TcpSocketLink.dispatchRequest(TcpSocketLink.java:1301)
at
com.caucho.network.listen.TcpSocketLink.handleRequest(TcpSocketLink.java:1257)
at
com.caucho.network.listen.TcpSocketLink.handleRequestsImpl(TcpSocketLink.java:1241)
at
com.caucho.network.listen.TcpSocketLink.handleRequests(TcpSocketLink.java:1149)
at
com.caucho.network.listen.TcpSocketLink.handleAcceptTaskImpl(TcpSocketLink.java:950)
at
com.caucho.network.listen.TcpSocketLink.handleAcceptTask(TcpSocketLink.java:902)
at
com.caucho.network.listen.ConnectionTask.runThread(ConnectionTask.java:114)
at
com.caucho.network.listen.ConnectionTask.run(ConnectionTask.java:93)
at
com.caucho.network.listen.SocketLinkThreadLauncher.handleTasks(SocketLinkThreadLauncher.java:169)
at
com.caucho.network.listen.TcpSocketResumeThread.run(TcpSocketResumeThread.java:62)
at
com.caucho.env.thread2.ResinThread2.runTasks(ResinThread2.java:173)
at
com.caucho.env.thread2.ResinThread2.run(ResinThread2.java:118)
>
> Karl
>
> On Fri, Sep 21, 2012 at 8:36 AM, Erlend Garåsen <e....@usit.uio.no> wrote:
>> On 21.09.12 14.07, Karl Wright wrote:
>>
>>> If you think the agents process is running, and yet you cannot delete
>>> a job, you should see stuff in the manifoldcf log that would indicate
>>> what the trouble likely is.
>>
>>
>> solr-test02 mcf-1 $ /www/var/data/mcf/mcf-1/agentctl start
>> Error: MCF Agent seems to be running already with PID 26310
>>
>> You're right. It has some problems with several documents:
>> WARN 2012-08-10 14:25:10,421 (Worker thread '4') - Pre-ingest service
>> interruption reported for job 1335434066944 connection 'web_crawler': Http
>> response temporary error on
>> 'http://www.apollon.uio.no/vis/art/1998/3/intervju': 503
>>
>> So I will ignore this at the moment and try to test a little bit more with
>> other parts of MCF.
>>
>> Erlend
>>
>>
>> --
>> Erlend Garåsen
>> Center for Information Technology Services
>> University of Oslo
>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
--
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Karl Wright <da...@gmail.com>.
A temporary error should not block a (non running) job from getting
cleaned up. The job can't be deleted until it is stopped, and no
outstanding documents are being worked on.
How many documents are still listed for the job in the UI?
Karl
On Fri, Sep 21, 2012 at 8:36 AM, Erlend Garåsen <e....@usit.uio.no> wrote:
> On 21.09.12 14.07, Karl Wright wrote:
>
>> If you think the agents process is running, and yet you cannot delete
>> a job, you should see stuff in the manifoldcf log that would indicate
>> what the trouble likely is.
>
>
> solr-test02 mcf-1 $ /www/var/data/mcf/mcf-1/agentctl start
> Error: MCF Agent seems to be running already with PID 26310
>
> You're right. It has some problems with several documents:
> WARN 2012-08-10 14:25:10,421 (Worker thread '4') - Pre-ingest service
> interruption reported for job 1335434066944 connection 'web_crawler': Http
> response temporary error on
> 'http://www.apollon.uio.no/vis/art/1998/3/intervju': 503
>
> So I will ignore this at the moment and try to test a little bit more with
> other parts of MCF.
>
> Erlend
>
>
> --
> Erlend Garåsen
> Center for Information Technology Services
> University of Oslo
> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Erlend Garåsen <e....@usit.uio.no>.
On 21.09.12 14.07, Karl Wright wrote:
> If you think the agents process is running, and yet you cannot delete
> a job, you should see stuff in the manifoldcf log that would indicate
> what the trouble likely is.
solr-test02 mcf-1 $ /www/var/data/mcf/mcf-1/agentctl start
Error: MCF Agent seems to be running already with PID 26310
You're right. It has some problems with several documents:
WARN 2012-08-10 14:25:10,421 (Worker thread '4') - Pre-ingest service
interruption reported for job 1335434066944 connection 'web_crawler':
Http response temporary error on
'http://www.apollon.uio.no/vis/art/1998/3/intervju': 503
So I will ignore this at the moment and try to test a little bit more
with other parts of MCF.
Erlend
--
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Karl Wright <da...@gmail.com>.
When a job is in the process of being deleted, it will display as
"cleaning up". The document count will go to zero and then the job
will be removed.
If you shut down the agents process, then of course none of this
happens. The document count stays the same and the job does not go
away, and the status page says "cleaning up" indefinitely.
If you think the agents process is running, and yet you cannot delete
a job, you should see stuff in the manifoldcf log that would indicate
what the trouble likely is.
Karl
On Fri, Sep 21, 2012 at 7:21 AM, Erlend Garåsen <e....@usit.uio.no> wrote:
>
> I have deployed this version on a development server running Resin. The only
> thing I have experienced so far is that I cannot delete any connections.
>
> Here's the following error when I try to delete the Solr Output connection:
> Can't delete output connection 'Solr': existing entities refer to it
>
> I have no authority connections, but I do have two repository connections.
> When I try to delete my file system connection, I get the following error:
> Can't delete repository connection 'file system': existing jobs refer to it
>
> But I *have* deleted my jobs! When I check the job statuses, it is "cleaning
> up". Start time and end time says "not started" / "never run".
>
> My connections were created on a previous version of MCF. The settings were
> imported by using the ImportConfiguration tool.
>
> Otherwise, everything seems to work as expected. I have tested:
> - Deployment on Resin 4 on Linux
> - ant test on OS X.
> - My new file encryption function.
>
> I'll wait with my vote until the above behaviour is explained.
>
> Erlend
>
>
>
> On 21.09.12 02.46, Karl Wright wrote:
>>
>> Please vote +1 to release ManifoldCF 1.0, RC0. The release artifact
>> can be found at:
>>
>> http://people.apache.org/~kwright/apache-manifoldcf-1.0
>>
>> There is also an SVN tag at:
>>
>> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.0-RC0
>>
>> Karl
>>
>
>
> --
> Erlend Garåsen
> Center for Information Technology Services
> University of Oslo
> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Erlend Garåsen <e....@usit.uio.no>.
I have deployed this version on a development server running Resin. The
only thing I have experienced so far is that I cannot delete any
connections.
Here's the following error when I try to delete the Solr Output connection:
Can't delete output connection 'Solr': existing entities refer to it
I have no authority connections, but I do have two repository
connections. When I try to delete my file system connection, I get the
following error:
Can't delete repository connection 'file system': existing jobs refer to it
But I *have* deleted my jobs! When I check the job statuses, it is
"cleaning up". Start time and end time says "not started" / "never run".
My connections were created on a previous version of MCF. The settings
were imported by using the ImportConfiguration tool.
Otherwise, everything seems to work as expected. I have tested:
- Deployment on Resin 4 on Linux
- ant test on OS X.
- My new file encryption function.
I'll wait with my vote until the above behaviour is explained.
Erlend
On 21.09.12 02.46, Karl Wright wrote:
> Please vote +1 to release ManifoldCF 1.0, RC0. The release artifact
> can be found at:
>
> http://people.apache.org/~kwright/apache-manifoldcf-1.0
>
> There is also an SVN tag at:
>
> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.0-RC0
>
> Karl
>
--
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC0
Posted by Piergiorgio Lucidi <pi...@apache.org>.
- Tried to use Alfresco and CMIS with success
- checked signatures
+1 from me
Piergiorgio
2012/9/21 Karl Wright <da...@gmail.com>
> Please vote +1 to release ManifoldCF 1.0, RC0. The release artifact
> can be found at:
>
> http://people.apache.org/~kwright/apache-manifoldcf-1.0
>
> There is also an SVN tag at:
>
> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.0-RC0
>
> Karl
>
> --
> Piergiorgio Lucidi
> http://www.open4dev.com
>
>