You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by Karl Wright <da...@gmail.com> on 2013/01/28 23:09:32 UTC

[VOTE] Release Apache ManifoldCF 1.1, RC7

Please vote on whether or not to release ManifoldCF 1.1, RC7.

The release artifact can be found at:

http://people.apache.org/~kwright/apache-manifoldcf-1.1

There is a tag at:

https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.1-RC7

This release candidate fixes a packaging problem for wars similar to
CONNECTORS-619.  It also fixes a problem with the CMIS connector
and another SolrJ-related issue (CONNECTORS-622 and CONNECTORS-623).

This release candidate provides a better workaround for
CONNECTORS-616 than RC5.  It also fixes CONNECTORS-617.

This release candidate fixes one problem since RC4, which is
the inconfigurability of the commit action path for Solr commits in
the Solr connector.  This needed to be fixed to maintain backwards
compatibility.  CONNECTORS-621.

This release candidate fixes two problems since RC3.  The problems
were in the included jars for the multiprocess example (CONNECTORS-619)
and in connection leakage for JDBC handles (CONNECTORS-620).

This release candidate fixes one problem since RC2.  The problem is
CONNECTORS-618, which relates to MySQL performance.

This release candidate fixes one additional problem since RC1.  The
problem is CONNECTORS-616, and relates to Solr dropping connections
during
indexing.

This release candidate fixes two other problems since RC0, both
related to Solr 4.0.0 support.
- CONNECTORS-613: The version of Tika used in Solr 4.0.0 cannot
extract text unless told an accurate mime type.  While this is
probably a Tika bug, in this ticket we at least make sure a good guess
as to the mime type is sent to Solr.
- CONNECTORS-614: Fix logic having to do with releasing idle Solr
connections.  This shows up as socket timeout exceptions, because it
becomes very easy to exhaust the Solr application server's thread pool
when idle connections are not released in a timely way.

This release includes a significant amount of long-planned upgrading
and refactoring since Apache ManifoldCF 1.0.1, including:
- Port to HttpComponents from commons-httpclient
- Port to SolrJ from homegrown for the Solr connector, so that
SolrCloud is supported
- Improved NTLM support
- Partial Kerberos support
- Many other improvements, which are summarized in CHANGES.txt

Re: [VOTE] Release Apache ManifoldCF 1.1, RC7

Posted by Minoru Osuka <mi...@gmail.com>.
Hi,

- Ran all tests.
- Solr Output Connection worked.

+1 from me.

Minoru OSUKA

On Jan 29, 2013, at 7:09 AM, Karl Wright <da...@gmail.com> wrote:

> Please vote on whether or not to release ManifoldCF 1.1, RC7.
> 
> The release artifact can be found at:
> 
> http://people.apache.org/~kwright/apache-manifoldcf-1.1
> 
> There is a tag at:
> 
> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.1-RC7
> 
> This release candidate fixes a packaging problem for wars similar to
> CONNECTORS-619.  It also fixes a problem with the CMIS connector
> and another SolrJ-related issue (CONNECTORS-622 and CONNECTORS-623).
> 
> This release candidate provides a better workaround for
> CONNECTORS-616 than RC5.  It also fixes CONNECTORS-617.
> 
> This release candidate fixes one problem since RC4, which is
> the inconfigurability of the commit action path for Solr commits in
> the Solr connector.  This needed to be fixed to maintain backwards
> compatibility.  CONNECTORS-621.
> 
> This release candidate fixes two problems since RC3.  The problems
> were in the included jars for the multiprocess example (CONNECTORS-619)
> and in connection leakage for JDBC handles (CONNECTORS-620).
> 
> This release candidate fixes one problem since RC2.  The problem is
> CONNECTORS-618, which relates to MySQL performance.
> 
> This release candidate fixes one additional problem since RC1.  The
> problem is CONNECTORS-616, and relates to Solr dropping connections
> during
> indexing.
> 
> This release candidate fixes two other problems since RC0, both
> related to Solr 4.0.0 support.
> - CONNECTORS-613: The version of Tika used in Solr 4.0.0 cannot
> extract text unless told an accurate mime type.  While this is
> probably a Tika bug, in this ticket we at least make sure a good guess
> as to the mime type is sent to Solr.
> - CONNECTORS-614: Fix logic having to do with releasing idle Solr
> connections.  This shows up as socket timeout exceptions, because it
> becomes very easy to exhaust the Solr application server's thread pool
> when idle connections are not released in a timely way.
> 
> This release includes a significant amount of long-planned upgrading
> and refactoring since Apache ManifoldCF 1.0.1, including:
> - Port to HttpComponents from commons-httpclient
> - Port to SolrJ from homegrown for the Solr connector, so that
> SolrCloud is supported
> - Improved NTLM support
> - Partial Kerberos support
> - Many other improvements, which are summarized in CHANGES.txt

Re: [VOTE] Release Apache ManifoldCF 1.1, RC7

Posted by Erlend Garåsen <e....@usit.uio.no>.
Difficult to say what caused these problems, but I have now deployed RC8 
which works well on Resin. I just have a couple of more tests to do, so 
I will give my final vote within a couple of hours.

Erlend


On 29.01.13 17.41, Karl Wright wrote:
> I ran the multiprocess example, with httpmime.jar just as we deliver
> it in the connector-lib directory, and I did not see this issue.  It
> is almost certainly configuration, seems likely.
>
> Karl
>
> On Tue, Jan 29, 2013 at 11:26 AM, Erlend Garåsen
> <e....@usit.uio.no> wrote:
>>
>> I have to run now, but I will investigate this further. BTW, I have the
>> following in my lib folder, so it should work:
>> httpmime.jar
>>
>> I did not see this yesterday when I was testing RC6 with Resin. The
>> difference now is that the crawler just fetches and fetches, but nothing
>> gets posted to Solr.
>>
>> I hop it is me who have misconfigured something, but I will get back to this
>> as soon as possible.
>>
>> FATAL 2013-01-29 17:19:17,609 (Worker thread '17') - Error tossed:
>> org/apache/http/entity/mime/content/ContentBody
>> java.lang.NoClassDefFoundError:
>> org/apache/http/entity/mime/content/ContentBody
>>          at
>> org.apache.manifoldcf.agents.output.solr.HttpPoster.<init>(HttpPoster.java:246)
>>          at
>> org.apache.manifoldcf.agents.output.solr.SolrConnector.getSession(SolrConnector.java:256)
>>          at
>> org.apache.manifoldcf.agents.output.solr.SolrConnector.removeDocument(SolrConnector.java:629)
>>          at
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.removeDocument(IncrementalIngester.java:1598)
>>          at
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:469)
>>          at
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:370)
>>          at
>> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1651)
>>          at
>> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.deleteDocument(WorkerThread.java:1672)
>>          at
>> org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.processDocuments(WebcrawlerConnector.java:1445)
>>          at
>> org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423)
>>          at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:551)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.http.entity.mime.content.ContentBody
>>          at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>          at java.security.AccessController.doPrivileged(Native Method)
>>          at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>          at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>          at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:627)
>>          at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>          ... 11 more
>>
>> Erlend
>>
>>
>> On 28.01.13 23.09, Karl Wright wrote:
>>>
>>> Please vote on whether or not to release ManifoldCF 1.1, RC7.
>>>
>>> The release artifact can be found at:
>>>
>>> http://people.apache.org/~kwright/apache-manifoldcf-1.1
>>>
>>> There is a tag at:
>>>
>>> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.1-RC7
>>>
>>> This release candidate fixes a packaging problem for wars similar to
>>> CONNECTORS-619.  It also fixes a problem with the CMIS connector
>>> and another SolrJ-related issue (CONNECTORS-622 and CONNECTORS-623).
>>>
>>> This release candidate provides a better workaround for
>>> CONNECTORS-616 than RC5.  It also fixes CONNECTORS-617.
>>>
>>> This release candidate fixes one problem since RC4, which is
>>> the inconfigurability of the commit action path for Solr commits in
>>> the Solr connector.  This needed to be fixed to maintain backwards
>>> compatibility.  CONNECTORS-621.
>>>
>>> This release candidate fixes two problems since RC3.  The problems
>>> were in the included jars for the multiprocess example (CONNECTORS-619)
>>> and in connection leakage for JDBC handles (CONNECTORS-620).
>>>
>>> This release candidate fixes one problem since RC2.  The problem is
>>> CONNECTORS-618, which relates to MySQL performance.
>>>
>>> This release candidate fixes one additional problem since RC1.  The
>>> problem is CONNECTORS-616, and relates to Solr dropping connections
>>> during
>>> indexing.
>>>
>>> This release candidate fixes two other problems since RC0, both
>>> related to Solr 4.0.0 support.
>>> - CONNECTORS-613: The version of Tika used in Solr 4.0.0 cannot
>>> extract text unless told an accurate mime type.  While this is
>>> probably a Tika bug, in this ticket we at least make sure a good guess
>>> as to the mime type is sent to Solr.
>>> - CONNECTORS-614: Fix logic having to do with releasing idle Solr
>>> connections.  This shows up as socket timeout exceptions, because it
>>> becomes very easy to exhaust the Solr application server's thread pool
>>> when idle connections are not released in a timely way.
>>>
>>> This release includes a significant amount of long-planned upgrading
>>> and refactoring since Apache ManifoldCF 1.0.1, including:
>>> - Port to HttpComponents from commons-httpclient
>>> - Port to SolrJ from homegrown for the Solr connector, so that
>>> SolrCloud is supported
>>> - Improved NTLM support
>>> - Partial Kerberos support
>>> - Many other improvements, which are summarized in CHANGES.txt
>>>
>>
>>
>> --
>> Erlend Garåsen
>> Center for Information Technology Services
>> University of Oslo
>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050


-- 
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050

[CANCEL][VOTE] Release Apache ManifoldCF 1.1, RC7

Posted by Karl Wright <da...@gmail.com>.
It looks like CONNECTORS-627 is going to make us spin another release candidate.

Karl


On Tue, Jan 29, 2013 at 11:41 AM, Karl Wright <da...@gmail.com> wrote:
> I ran the multiprocess example, with httpmime.jar just as we deliver
> it in the connector-lib directory, and I did not see this issue.  It
> is almost certainly configuration, seems likely.
>
> Karl
>
> On Tue, Jan 29, 2013 at 11:26 AM, Erlend Garåsen
> <e....@usit.uio.no> wrote:
>>
>> I have to run now, but I will investigate this further. BTW, I have the
>> following in my lib folder, so it should work:
>> httpmime.jar
>>
>> I did not see this yesterday when I was testing RC6 with Resin. The
>> difference now is that the crawler just fetches and fetches, but nothing
>> gets posted to Solr.
>>
>> I hop it is me who have misconfigured something, but I will get back to this
>> as soon as possible.
>>
>> FATAL 2013-01-29 17:19:17,609 (Worker thread '17') - Error tossed:
>> org/apache/http/entity/mime/content/ContentBody
>> java.lang.NoClassDefFoundError:
>> org/apache/http/entity/mime/content/ContentBody
>>         at
>> org.apache.manifoldcf.agents.output.solr.HttpPoster.<init>(HttpPoster.java:246)
>>         at
>> org.apache.manifoldcf.agents.output.solr.SolrConnector.getSession(SolrConnector.java:256)
>>         at
>> org.apache.manifoldcf.agents.output.solr.SolrConnector.removeDocument(SolrConnector.java:629)
>>         at
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.removeDocument(IncrementalIngester.java:1598)
>>         at
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:469)
>>         at
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:370)
>>         at
>> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1651)
>>         at
>> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.deleteDocument(WorkerThread.java:1672)
>>         at
>> org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.processDocuments(WebcrawlerConnector.java:1445)
>>         at
>> org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423)
>>         at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:551)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.http.entity.mime.content.ContentBody
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>         at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:627)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>         ... 11 more
>>
>> Erlend
>>
>>
>> On 28.01.13 23.09, Karl Wright wrote:
>>>
>>> Please vote on whether or not to release ManifoldCF 1.1, RC7.
>>>
>>> The release artifact can be found at:
>>>
>>> http://people.apache.org/~kwright/apache-manifoldcf-1.1
>>>
>>> There is a tag at:
>>>
>>> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.1-RC7
>>>
>>> This release candidate fixes a packaging problem for wars similar to
>>> CONNECTORS-619.  It also fixes a problem with the CMIS connector
>>> and another SolrJ-related issue (CONNECTORS-622 and CONNECTORS-623).
>>>
>>> This release candidate provides a better workaround for
>>> CONNECTORS-616 than RC5.  It also fixes CONNECTORS-617.
>>>
>>> This release candidate fixes one problem since RC4, which is
>>> the inconfigurability of the commit action path for Solr commits in
>>> the Solr connector.  This needed to be fixed to maintain backwards
>>> compatibility.  CONNECTORS-621.
>>>
>>> This release candidate fixes two problems since RC3.  The problems
>>> were in the included jars for the multiprocess example (CONNECTORS-619)
>>> and in connection leakage for JDBC handles (CONNECTORS-620).
>>>
>>> This release candidate fixes one problem since RC2.  The problem is
>>> CONNECTORS-618, which relates to MySQL performance.
>>>
>>> This release candidate fixes one additional problem since RC1.  The
>>> problem is CONNECTORS-616, and relates to Solr dropping connections
>>> during
>>> indexing.
>>>
>>> This release candidate fixes two other problems since RC0, both
>>> related to Solr 4.0.0 support.
>>> - CONNECTORS-613: The version of Tika used in Solr 4.0.0 cannot
>>> extract text unless told an accurate mime type.  While this is
>>> probably a Tika bug, in this ticket we at least make sure a good guess
>>> as to the mime type is sent to Solr.
>>> - CONNECTORS-614: Fix logic having to do with releasing idle Solr
>>> connections.  This shows up as socket timeout exceptions, because it
>>> becomes very easy to exhaust the Solr application server's thread pool
>>> when idle connections are not released in a timely way.
>>>
>>> This release includes a significant amount of long-planned upgrading
>>> and refactoring since Apache ManifoldCF 1.0.1, including:
>>> - Port to HttpComponents from commons-httpclient
>>> - Port to SolrJ from homegrown for the Solr connector, so that
>>> SolrCloud is supported
>>> - Improved NTLM support
>>> - Partial Kerberos support
>>> - Many other improvements, which are summarized in CHANGES.txt
>>>
>>
>>
>> --
>> Erlend Garåsen
>> Center for Information Technology Services
>> University of Oslo
>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050

Re: [VOTE] Release Apache ManifoldCF 1.1, RC7

Posted by Karl Wright <da...@gmail.com>.
I ran the multiprocess example, with httpmime.jar just as we deliver
it in the connector-lib directory, and I did not see this issue.  It
is almost certainly configuration, seems likely.

Karl

On Tue, Jan 29, 2013 at 11:26 AM, Erlend Garåsen
<e....@usit.uio.no> wrote:
>
> I have to run now, but I will investigate this further. BTW, I have the
> following in my lib folder, so it should work:
> httpmime.jar
>
> I did not see this yesterday when I was testing RC6 with Resin. The
> difference now is that the crawler just fetches and fetches, but nothing
> gets posted to Solr.
>
> I hop it is me who have misconfigured something, but I will get back to this
> as soon as possible.
>
> FATAL 2013-01-29 17:19:17,609 (Worker thread '17') - Error tossed:
> org/apache/http/entity/mime/content/ContentBody
> java.lang.NoClassDefFoundError:
> org/apache/http/entity/mime/content/ContentBody
>         at
> org.apache.manifoldcf.agents.output.solr.HttpPoster.<init>(HttpPoster.java:246)
>         at
> org.apache.manifoldcf.agents.output.solr.SolrConnector.getSession(SolrConnector.java:256)
>         at
> org.apache.manifoldcf.agents.output.solr.SolrConnector.removeDocument(SolrConnector.java:629)
>         at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.removeDocument(IncrementalIngester.java:1598)
>         at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:469)
>         at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:370)
>         at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1651)
>         at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.deleteDocument(WorkerThread.java:1672)
>         at
> org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.processDocuments(WebcrawlerConnector.java:1445)
>         at
> org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423)
>         at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:551)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.http.entity.mime.content.ContentBody
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>         at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:627)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>         ... 11 more
>
> Erlend
>
>
> On 28.01.13 23.09, Karl Wright wrote:
>>
>> Please vote on whether or not to release ManifoldCF 1.1, RC7.
>>
>> The release artifact can be found at:
>>
>> http://people.apache.org/~kwright/apache-manifoldcf-1.1
>>
>> There is a tag at:
>>
>> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.1-RC7
>>
>> This release candidate fixes a packaging problem for wars similar to
>> CONNECTORS-619.  It also fixes a problem with the CMIS connector
>> and another SolrJ-related issue (CONNECTORS-622 and CONNECTORS-623).
>>
>> This release candidate provides a better workaround for
>> CONNECTORS-616 than RC5.  It also fixes CONNECTORS-617.
>>
>> This release candidate fixes one problem since RC4, which is
>> the inconfigurability of the commit action path for Solr commits in
>> the Solr connector.  This needed to be fixed to maintain backwards
>> compatibility.  CONNECTORS-621.
>>
>> This release candidate fixes two problems since RC3.  The problems
>> were in the included jars for the multiprocess example (CONNECTORS-619)
>> and in connection leakage for JDBC handles (CONNECTORS-620).
>>
>> This release candidate fixes one problem since RC2.  The problem is
>> CONNECTORS-618, which relates to MySQL performance.
>>
>> This release candidate fixes one additional problem since RC1.  The
>> problem is CONNECTORS-616, and relates to Solr dropping connections
>> during
>> indexing.
>>
>> This release candidate fixes two other problems since RC0, both
>> related to Solr 4.0.0 support.
>> - CONNECTORS-613: The version of Tika used in Solr 4.0.0 cannot
>> extract text unless told an accurate mime type.  While this is
>> probably a Tika bug, in this ticket we at least make sure a good guess
>> as to the mime type is sent to Solr.
>> - CONNECTORS-614: Fix logic having to do with releasing idle Solr
>> connections.  This shows up as socket timeout exceptions, because it
>> becomes very easy to exhaust the Solr application server's thread pool
>> when idle connections are not released in a timely way.
>>
>> This release includes a significant amount of long-planned upgrading
>> and refactoring since Apache ManifoldCF 1.0.1, including:
>> - Port to HttpComponents from commons-httpclient
>> - Port to SolrJ from homegrown for the Solr connector, so that
>> SolrCloud is supported
>> - Improved NTLM support
>> - Partial Kerberos support
>> - Many other improvements, which are summarized in CHANGES.txt
>>
>
>
> --
> Erlend Garåsen
> Center for Information Technology Services
> University of Oslo
> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050

Re: [VOTE] Release Apache ManifoldCF 1.1, RC7

Posted by Erlend Garåsen <e....@usit.uio.no>.
I have to run now, but I will investigate this further. BTW, I have the 
following in my lib folder, so it should work:
httpmime.jar

I did not see this yesterday when I was testing RC6 with Resin. The 
difference now is that the crawler just fetches and fetches, but nothing 
gets posted to Solr.

I hop it is me who have misconfigured something, but I will get back to 
this as soon as possible.

FATAL 2013-01-29 17:19:17,609 (Worker thread '17') - Error tossed: 
org/apache/http/entity/mime/content/ContentBody
java.lang.NoClassDefFoundError: 
org/apache/http/entity/mime/content/ContentBody
	at 
org.apache.manifoldcf.agents.output.solr.HttpPoster.<init>(HttpPoster.java:246)
	at 
org.apache.manifoldcf.agents.output.solr.SolrConnector.getSession(SolrConnector.java:256)
	at 
org.apache.manifoldcf.agents.output.solr.SolrConnector.removeDocument(SolrConnector.java:629)
	at 
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.removeDocument(IncrementalIngester.java:1598)
	at 
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:469)
	at 
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:370)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1651)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.deleteDocument(WorkerThread.java:1672)
	at 
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.processDocuments(WebcrawlerConnector.java:1445)
	at 
org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423)
	at 
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:551)
Caused by: java.lang.ClassNotFoundException: 
org.apache.http.entity.mime.content.ContentBody
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:627)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
	... 11 more

Erlend

On 28.01.13 23.09, Karl Wright wrote:
> Please vote on whether or not to release ManifoldCF 1.1, RC7.
>
> The release artifact can be found at:
>
> http://people.apache.org/~kwright/apache-manifoldcf-1.1
>
> There is a tag at:
>
> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.1-RC7
>
> This release candidate fixes a packaging problem for wars similar to
> CONNECTORS-619.  It also fixes a problem with the CMIS connector
> and another SolrJ-related issue (CONNECTORS-622 and CONNECTORS-623).
>
> This release candidate provides a better workaround for
> CONNECTORS-616 than RC5.  It also fixes CONNECTORS-617.
>
> This release candidate fixes one problem since RC4, which is
> the inconfigurability of the commit action path for Solr commits in
> the Solr connector.  This needed to be fixed to maintain backwards
> compatibility.  CONNECTORS-621.
>
> This release candidate fixes two problems since RC3.  The problems
> were in the included jars for the multiprocess example (CONNECTORS-619)
> and in connection leakage for JDBC handles (CONNECTORS-620).
>
> This release candidate fixes one problem since RC2.  The problem is
> CONNECTORS-618, which relates to MySQL performance.
>
> This release candidate fixes one additional problem since RC1.  The
> problem is CONNECTORS-616, and relates to Solr dropping connections
> during
> indexing.
>
> This release candidate fixes two other problems since RC0, both
> related to Solr 4.0.0 support.
> - CONNECTORS-613: The version of Tika used in Solr 4.0.0 cannot
> extract text unless told an accurate mime type.  While this is
> probably a Tika bug, in this ticket we at least make sure a good guess
> as to the mime type is sent to Solr.
> - CONNECTORS-614: Fix logic having to do with releasing idle Solr
> connections.  This shows up as socket timeout exceptions, because it
> becomes very easy to exhaust the Solr application server's thread pool
> when idle connections are not released in a timely way.
>
> This release includes a significant amount of long-planned upgrading
> and refactoring since Apache ManifoldCF 1.0.1, including:
> - Port to HttpComponents from commons-httpclient
> - Port to SolrJ from homegrown for the Solr connector, so that
> SolrCloud is supported
> - Improved NTLM support
> - Partial Kerberos support
> - Many other improvements, which are summarized in CHANGES.txt
>


-- 
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050

Re: [VOTE] Release Apache ManifoldCF 1.1, RC7

Posted by Karl Wright <da...@gmail.com>.
At first glance this looks like a bug in SolrJ.  I also seem to recall
seeing this mentioned on the dev list.

If this is critical to anyone, AND we can confirm that it is fixed in
SolrJ 4.1, maybe we should consider going with the 4.1 jar for our
release.  Can you confirm this?

Karl

On Tue, Jan 29, 2013 at 11:04 AM, Shinichiro Abe
<sh...@gmail.com> wrote:
> I found another problem.
>
> When setting "single server" at Solr type in Solr connection, the connection status works fine on both Solr 4.0 and Solr 4.1.
> When setting "SolrCloud" at Solr type in Solr connection, the connection status works fine on Solr 4.0,
> but the connection status DOES NOT work on Solr 4.1:
> I will look into this tomorrow.
>
> 478690 [qtp1162992854-174] WARN org.eclipse.jetty.servlet.ServletHandler - /mcf-crawler-ui/execute.jsp
> org.apache.jasper.JasperException: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map
>         at org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:502)
>         at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:430)
>         at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
>         at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>         at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:547)
>         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:480)
>         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>         at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:520)
>         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
>         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:941)
>         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:409)
>         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
>         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:875)
>         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>         at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:288)
>         at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:115)
>         at org.apache.jasper.runtime.PageContextImpl.doForward(PageContextImpl.java:709)
>         at org.apache.jasper.runtime.PageContextImpl.forward(PageContextImpl.java:680)
>         at org.apache.jsp.execute_jsp._jspService(execute_jsp.java:734)
>         at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>         at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:388)
>         at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
>         at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>         at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:547)
>         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:480)
>         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>         at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:520)
>         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
>         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:941)
>         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:409)
>         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
>         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:875)
>         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>         at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
>         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
>         at org.eclipse.jetty.server.Server.handle(Server.java:349)
>         at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:441)
>         at org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:936)
>         at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:801)
>         at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:224)
>         at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:51)
>         at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:586)
>         at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:44)
>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:598)
>         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:533)
>         at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map
>         at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:291)
>         at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:263)
>         at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:274)
>         at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:142)
>         at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:165)
>         at org.apache.manifoldcf.agents.output.solr.HttpPoster$SolrPing.process(HttpPoster.java:1197)
>         at org.apache.manifoldcf.agents.output.solr.HttpPoster$StatusThread.run(HttpPoster.java:1090)
>
> Shinichiro Abe
>
> On 2013/01/29, at 7:09, Karl Wright wrote:
>
>> Please vote on whether or not to release ManifoldCF 1.1, RC7.
>>
>> The release artifact can be found at:
>>
>> http://people.apache.org/~kwright/apache-manifoldcf-1.1
>>
>> There is a tag at:
>>
>> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.1-RC7
>>
>> This release candidate fixes a packaging problem for wars similar to
>> CONNECTORS-619.  It also fixes a problem with the CMIS connector
>> and another SolrJ-related issue (CONNECTORS-622 and CONNECTORS-623).
>>
>> This release candidate provides a better workaround for
>> CONNECTORS-616 than RC5.  It also fixes CONNECTORS-617.
>>
>> This release candidate fixes one problem since RC4, which is
>> the inconfigurability of the commit action path for Solr commits in
>> the Solr connector.  This needed to be fixed to maintain backwards
>> compatibility.  CONNECTORS-621.
>>
>> This release candidate fixes two problems since RC3.  The problems
>> were in the included jars for the multiprocess example (CONNECTORS-619)
>> and in connection leakage for JDBC handles (CONNECTORS-620).
>>
>> This release candidate fixes one problem since RC2.  The problem is
>> CONNECTORS-618, which relates to MySQL performance.
>>
>> This release candidate fixes one additional problem since RC1.  The
>> problem is CONNECTORS-616, and relates to Solr dropping connections
>> during
>> indexing.
>>
>> This release candidate fixes two other problems since RC0, both
>> related to Solr 4.0.0 support.
>> - CONNECTORS-613: The version of Tika used in Solr 4.0.0 cannot
>> extract text unless told an accurate mime type.  While this is
>> probably a Tika bug, in this ticket we at least make sure a good guess
>> as to the mime type is sent to Solr.
>> - CONNECTORS-614: Fix logic having to do with releasing idle Solr
>> connections.  This shows up as socket timeout exceptions, because it
>> becomes very easy to exhaust the Solr application server's thread pool
>> when idle connections are not released in a timely way.
>>
>> This release includes a significant amount of long-planned upgrading
>> and refactoring since Apache ManifoldCF 1.0.1, including:
>> - Port to HttpComponents from commons-httpclient
>> - Port to SolrJ from homegrown for the Solr connector, so that
>> SolrCloud is supported
>> - Improved NTLM support
>> - Partial Kerberos support
>> - Many other improvements, which are summarized in CHANGES.txt
>

Re: [VOTE] Release Apache ManifoldCF 1.1, RC7

Posted by Shinichiro Abe <sh...@gmail.com>.
I found another problem.

When setting "single server" at Solr type in Solr connection, the connection status works fine on both Solr 4.0 and Solr 4.1.
When setting "SolrCloud" at Solr type in Solr connection, the connection status works fine on Solr 4.0,
but the connection status DOES NOT work on Solr 4.1:
I will look into this tomorrow.

478690 [qtp1162992854-174] WARN org.eclipse.jetty.servlet.ServletHandler - /mcf-crawler-ui/execute.jsp
org.apache.jasper.JasperException: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map
	at org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:502)
	at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:430)
	at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
	at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:547)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:480)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:520)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:941)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:409)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:875)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:288)
	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:115)
	at org.apache.jasper.runtime.PageContextImpl.doForward(PageContextImpl.java:709)
	at org.apache.jasper.runtime.PageContextImpl.forward(PageContextImpl.java:680)
	at org.apache.jsp.execute_jsp._jspService(execute_jsp.java:734)
	at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:388)
	at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
	at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:547)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:480)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:520)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:941)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:409)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:875)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
	at org.eclipse.jetty.server.Server.handle(Server.java:349)
	at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:441)
	at org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:936)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:801)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:224)
	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:51)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:586)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:44)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:598)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:533)
	at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map
	at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:291)
	at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:263)
	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:274)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:142)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:165)
	at org.apache.manifoldcf.agents.output.solr.HttpPoster$SolrPing.process(HttpPoster.java:1197)
	at org.apache.manifoldcf.agents.output.solr.HttpPoster$StatusThread.run(HttpPoster.java:1090)

Shinichiro Abe

On 2013/01/29, at 7:09, Karl Wright wrote:

> Please vote on whether or not to release ManifoldCF 1.1, RC7.
> 
> The release artifact can be found at:
> 
> http://people.apache.org/~kwright/apache-manifoldcf-1.1
> 
> There is a tag at:
> 
> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.1-RC7
> 
> This release candidate fixes a packaging problem for wars similar to
> CONNECTORS-619.  It also fixes a problem with the CMIS connector
> and another SolrJ-related issue (CONNECTORS-622 and CONNECTORS-623).
> 
> This release candidate provides a better workaround for
> CONNECTORS-616 than RC5.  It also fixes CONNECTORS-617.
> 
> This release candidate fixes one problem since RC4, which is
> the inconfigurability of the commit action path for Solr commits in
> the Solr connector.  This needed to be fixed to maintain backwards
> compatibility.  CONNECTORS-621.
> 
> This release candidate fixes two problems since RC3.  The problems
> were in the included jars for the multiprocess example (CONNECTORS-619)
> and in connection leakage for JDBC handles (CONNECTORS-620).
> 
> This release candidate fixes one problem since RC2.  The problem is
> CONNECTORS-618, which relates to MySQL performance.
> 
> This release candidate fixes one additional problem since RC1.  The
> problem is CONNECTORS-616, and relates to Solr dropping connections
> during
> indexing.
> 
> This release candidate fixes two other problems since RC0, both
> related to Solr 4.0.0 support.
> - CONNECTORS-613: The version of Tika used in Solr 4.0.0 cannot
> extract text unless told an accurate mime type.  While this is
> probably a Tika bug, in this ticket we at least make sure a good guess
> as to the mime type is sent to Solr.
> - CONNECTORS-614: Fix logic having to do with releasing idle Solr
> connections.  This shows up as socket timeout exceptions, because it
> becomes very easy to exhaust the Solr application server's thread pool
> when idle connections are not released in a timely way.
> 
> This release includes a significant amount of long-planned upgrading
> and refactoring since Apache ManifoldCF 1.0.1, including:
> - Port to HttpComponents from commons-httpclient
> - Port to SolrJ from homegrown for the Solr connector, so that
> SolrCloud is supported
> - Improved NTLM support
> - Partial Kerberos support
> - Many other improvements, which are summarized in CHANGES.txt


Re: [VOTE] Release Apache ManifoldCF 1.1, RC7

Posted by Shinichiro Abe <sh...@gmail.com>.
I could confirm these pages are retried and ingested eventually.
I also think it is OK. So I'm +1 for RC7.

Shinichiro Abe

On 2013/01/29, at 23:04, Karl Wright wrote:

> As long as the fetch is retried, and is eventually successful, I think
> this is OK.  Can you confirm this?
> 
> Karl
> 
> 
> On Tue, Jan 29, 2013 at 9:01 AM, Shinichiro Abe
> <sh...@gmail.com> wrote:
>> Hi,
>> 
>> I'm using Web connector in RC7, I saw the symptom which I didn't see in MCF 1.0.1.
>> Simple History is showing sometimes:
>> Result Code -1, "The target server failed to respond" at fetch activity.
>> 
>> Now I'm using Web connector in MCF 1.0.1, but I can not see this results at all.
>> 
>> Simple History is showing:
>> Start Time      Activity        Identifier      Result Code     Bytes   Time    Result Description
>> 01-29-2013 22:29:26.999 fetch   http://www.rondhuit.com/technologies/apache-lucene-apache-solr.html
>> -1      0       1       The target server failed to respond
>> 01-29-2013 22:29:06.999 fetch   http://www.rondhuit.com/?p=836
>> -1      0       2       The target server failed to respond
>> 
>> manifoldcf.log is showing:
>> WARN 2013-01-29 22:29:07,003 (Worker thread '27') - Pre-ingest service interruption reported for job 1359465187706 connection '1': Timed out waiting for response for 'http://www.rondhuit.com/?p=836': The target server failed to respond
>> WARN 2013-01-29 22:29:27,004 (Worker thread '41') - Pre-ingest service interruption reported for job 1359465187706 connection '1': Timed out waiting for response for 'http://www.rondhuit.com/technologies/apache-lucene-apache-solr.html': The target server failed to respond
>> 
>> I'm +1 if this problem is not critical.
>> I've already checked Filesystem, JCIFS, JDBC and SharePoint connector.
>> 
>> Regards,
>> Shinichiro Abe
>> 
>> 
>> 
>> On 2013/01/29, at 9:07, Karl Wright wrote:
>> 
>>> - Ran all tests
>>> - Ran multiprocess example, and created a solr output connection
>>> 
>>> +1 from me.
>>> 
>>> Karl
>>> 
>>> On Mon, Jan 28, 2013 at 5:09 PM, Karl Wright <da...@gmail.com> wrote:
>>>> Please vote on whether or not to release ManifoldCF 1.1, RC7.
>>>> 
>>>> The release artifact can be found at:
>>>> 
>>>> http://people.apache.org/~kwright/apache-manifoldcf-1.1
>>>> 
>>>> There is a tag at:
>>>> 
>>>> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.1-RC7
>>>> 
>>>> This release candidate fixes a packaging problem for wars similar to
>>>> CONNECTORS-619.  It also fixes a problem with the CMIS connector
>>>> and another SolrJ-related issue (CONNECTORS-622 and CONNECTORS-623).
>>>> 
>>>> This release candidate provides a better workaround for
>>>> CONNECTORS-616 than RC5.  It also fixes CONNECTORS-617.
>>>> 
>>>> This release candidate fixes one problem since RC4, which is
>>>> the inconfigurability of the commit action path for Solr commits in
>>>> the Solr connector.  This needed to be fixed to maintain backwards
>>>> compatibility.  CONNECTORS-621.
>>>> 
>>>> This release candidate fixes two problems since RC3.  The problems
>>>> were in the included jars for the multiprocess example (CONNECTORS-619)
>>>> and in connection leakage for JDBC handles (CONNECTORS-620).
>>>> 
>>>> This release candidate fixes one problem since RC2.  The problem is
>>>> CONNECTORS-618, which relates to MySQL performance.
>>>> 
>>>> This release candidate fixes one additional problem since RC1.  The
>>>> problem is CONNECTORS-616, and relates to Solr dropping connections
>>>> during
>>>> indexing.
>>>> 
>>>> This release candidate fixes two other problems since RC0, both
>>>> related to Solr 4.0.0 support.
>>>> - CONNECTORS-613: The version of Tika used in Solr 4.0.0 cannot
>>>> extract text unless told an accurate mime type.  While this is
>>>> probably a Tika bug, in this ticket we at least make sure a good guess
>>>> as to the mime type is sent to Solr.
>>>> - CONNECTORS-614: Fix logic having to do with releasing idle Solr
>>>> connections.  This shows up as socket timeout exceptions, because it
>>>> becomes very easy to exhaust the Solr application server's thread pool
>>>> when idle connections are not released in a timely way.
>>>> 
>>>> This release includes a significant amount of long-planned upgrading
>>>> and refactoring since Apache ManifoldCF 1.0.1, including:
>>>> - Port to HttpComponents from commons-httpclient
>>>> - Port to SolrJ from homegrown for the Solr connector, so that
>>>> SolrCloud is supported
>>>> - Improved NTLM support
>>>> - Partial Kerberos support
>>>> - Many other improvements, which are summarized in CHANGES.txt
>> 


Re: [VOTE] Release Apache ManifoldCF 1.1, RC7

Posted by Karl Wright <da...@gmail.com>.
As long as the fetch is retried, and is eventually successful, I think
this is OK.  Can you confirm this?

Karl


On Tue, Jan 29, 2013 at 9:01 AM, Shinichiro Abe
<sh...@gmail.com> wrote:
> Hi,
>
> I'm using Web connector in RC7, I saw the symptom which I didn't see in MCF 1.0.1.
> Simple History is showing sometimes:
> Result Code -1, "The target server failed to respond" at fetch activity.
>
> Now I'm using Web connector in MCF 1.0.1, but I can not see this results at all.
>
> Simple History is showing:
> Start Time      Activity        Identifier      Result Code     Bytes   Time    Result Description
> 01-29-2013 22:29:26.999 fetch   http://www.rondhuit.com/technologies/apache-lucene-apache-solr.html
> -1      0       1       The target server failed to respond
> 01-29-2013 22:29:06.999 fetch   http://www.rondhuit.com/?p=836
> -1      0       2       The target server failed to respond
>
> manifoldcf.log is showing:
>  WARN 2013-01-29 22:29:07,003 (Worker thread '27') - Pre-ingest service interruption reported for job 1359465187706 connection '1': Timed out waiting for response for 'http://www.rondhuit.com/?p=836': The target server failed to respond
>  WARN 2013-01-29 22:29:27,004 (Worker thread '41') - Pre-ingest service interruption reported for job 1359465187706 connection '1': Timed out waiting for response for 'http://www.rondhuit.com/technologies/apache-lucene-apache-solr.html': The target server failed to respond
>
> I'm +1 if this problem is not critical.
> I've already checked Filesystem, JCIFS, JDBC and SharePoint connector.
>
> Regards,
> Shinichiro Abe
>
>
>
> On 2013/01/29, at 9:07, Karl Wright wrote:
>
>> - Ran all tests
>> - Ran multiprocess example, and created a solr output connection
>>
>> +1 from me.
>>
>> Karl
>>
>> On Mon, Jan 28, 2013 at 5:09 PM, Karl Wright <da...@gmail.com> wrote:
>>> Please vote on whether or not to release ManifoldCF 1.1, RC7.
>>>
>>> The release artifact can be found at:
>>>
>>> http://people.apache.org/~kwright/apache-manifoldcf-1.1
>>>
>>> There is a tag at:
>>>
>>> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.1-RC7
>>>
>>> This release candidate fixes a packaging problem for wars similar to
>>> CONNECTORS-619.  It also fixes a problem with the CMIS connector
>>> and another SolrJ-related issue (CONNECTORS-622 and CONNECTORS-623).
>>>
>>> This release candidate provides a better workaround for
>>> CONNECTORS-616 than RC5.  It also fixes CONNECTORS-617.
>>>
>>> This release candidate fixes one problem since RC4, which is
>>> the inconfigurability of the commit action path for Solr commits in
>>> the Solr connector.  This needed to be fixed to maintain backwards
>>> compatibility.  CONNECTORS-621.
>>>
>>> This release candidate fixes two problems since RC3.  The problems
>>> were in the included jars for the multiprocess example (CONNECTORS-619)
>>> and in connection leakage for JDBC handles (CONNECTORS-620).
>>>
>>> This release candidate fixes one problem since RC2.  The problem is
>>> CONNECTORS-618, which relates to MySQL performance.
>>>
>>> This release candidate fixes one additional problem since RC1.  The
>>> problem is CONNECTORS-616, and relates to Solr dropping connections
>>> during
>>> indexing.
>>>
>>> This release candidate fixes two other problems since RC0, both
>>> related to Solr 4.0.0 support.
>>> - CONNECTORS-613: The version of Tika used in Solr 4.0.0 cannot
>>> extract text unless told an accurate mime type.  While this is
>>> probably a Tika bug, in this ticket we at least make sure a good guess
>>> as to the mime type is sent to Solr.
>>> - CONNECTORS-614: Fix logic having to do with releasing idle Solr
>>> connections.  This shows up as socket timeout exceptions, because it
>>> becomes very easy to exhaust the Solr application server's thread pool
>>> when idle connections are not released in a timely way.
>>>
>>> This release includes a significant amount of long-planned upgrading
>>> and refactoring since Apache ManifoldCF 1.0.1, including:
>>> - Port to HttpComponents from commons-httpclient
>>> - Port to SolrJ from homegrown for the Solr connector, so that
>>> SolrCloud is supported
>>> - Improved NTLM support
>>> - Partial Kerberos support
>>> - Many other improvements, which are summarized in CHANGES.txt
>

Re: [VOTE] Release Apache ManifoldCF 1.1, RC7

Posted by Shinichiro Abe <sh...@gmail.com>.
Hi,

I'm using Web connector in RC7, I saw the symptom which I didn't see in MCF 1.0.1.
Simple History is showing sometimes: 
Result Code -1, "The target server failed to respond" at fetch activity.

Now I'm using Web connector in MCF 1.0.1, but I can not see this results at all.

Simple History is showing:
Start Time	Activity	Identifier	Result Code	Bytes	Time	Result Description
01-29-2013 22:29:26.999	fetch	http://www.rondhuit.com/technologies/apache-lucene-apache-solr.html
-1	0	1	The target server failed to respond
01-29-2013 22:29:06.999	fetch	http://www.rondhuit.com/?p=836
-1	0	2	The target server failed to respond

manifoldcf.log is showing:
 WARN 2013-01-29 22:29:07,003 (Worker thread '27') - Pre-ingest service interruption reported for job 1359465187706 connection '1': Timed out waiting for response for 'http://www.rondhuit.com/?p=836': The target server failed to respond
 WARN 2013-01-29 22:29:27,004 (Worker thread '41') - Pre-ingest service interruption reported for job 1359465187706 connection '1': Timed out waiting for response for 'http://www.rondhuit.com/technologies/apache-lucene-apache-solr.html': The target server failed to respond

I'm +1 if this problem is not critical.
I've already checked Filesystem, JCIFS, JDBC and SharePoint connector.

Regards,
Shinichiro Abe



On 2013/01/29, at 9:07, Karl Wright wrote:

> - Ran all tests
> - Ran multiprocess example, and created a solr output connection
> 
> +1 from me.
> 
> Karl
> 
> On Mon, Jan 28, 2013 at 5:09 PM, Karl Wright <da...@gmail.com> wrote:
>> Please vote on whether or not to release ManifoldCF 1.1, RC7.
>> 
>> The release artifact can be found at:
>> 
>> http://people.apache.org/~kwright/apache-manifoldcf-1.1
>> 
>> There is a tag at:
>> 
>> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.1-RC7
>> 
>> This release candidate fixes a packaging problem for wars similar to
>> CONNECTORS-619.  It also fixes a problem with the CMIS connector
>> and another SolrJ-related issue (CONNECTORS-622 and CONNECTORS-623).
>> 
>> This release candidate provides a better workaround for
>> CONNECTORS-616 than RC5.  It also fixes CONNECTORS-617.
>> 
>> This release candidate fixes one problem since RC4, which is
>> the inconfigurability of the commit action path for Solr commits in
>> the Solr connector.  This needed to be fixed to maintain backwards
>> compatibility.  CONNECTORS-621.
>> 
>> This release candidate fixes two problems since RC3.  The problems
>> were in the included jars for the multiprocess example (CONNECTORS-619)
>> and in connection leakage for JDBC handles (CONNECTORS-620).
>> 
>> This release candidate fixes one problem since RC2.  The problem is
>> CONNECTORS-618, which relates to MySQL performance.
>> 
>> This release candidate fixes one additional problem since RC1.  The
>> problem is CONNECTORS-616, and relates to Solr dropping connections
>> during
>> indexing.
>> 
>> This release candidate fixes two other problems since RC0, both
>> related to Solr 4.0.0 support.
>> - CONNECTORS-613: The version of Tika used in Solr 4.0.0 cannot
>> extract text unless told an accurate mime type.  While this is
>> probably a Tika bug, in this ticket we at least make sure a good guess
>> as to the mime type is sent to Solr.
>> - CONNECTORS-614: Fix logic having to do with releasing idle Solr
>> connections.  This shows up as socket timeout exceptions, because it
>> becomes very easy to exhaust the Solr application server's thread pool
>> when idle connections are not released in a timely way.
>> 
>> This release includes a significant amount of long-planned upgrading
>> and refactoring since Apache ManifoldCF 1.0.1, including:
>> - Port to HttpComponents from commons-httpclient
>> - Port to SolrJ from homegrown for the Solr connector, so that
>> SolrCloud is supported
>> - Improved NTLM support
>> - Partial Kerberos support
>> - Many other improvements, which are summarized in CHANGES.txt


Re: [VOTE] Release Apache ManifoldCF 1.1, RC7

Posted by Karl Wright <da...@gmail.com>.
- Ran all tests
- Ran multiprocess example, and created a solr output connection

+1 from me.

Karl

On Mon, Jan 28, 2013 at 5:09 PM, Karl Wright <da...@gmail.com> wrote:
> Please vote on whether or not to release ManifoldCF 1.1, RC7.
>
> The release artifact can be found at:
>
> http://people.apache.org/~kwright/apache-manifoldcf-1.1
>
> There is a tag at:
>
> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.1-RC7
>
> This release candidate fixes a packaging problem for wars similar to
> CONNECTORS-619.  It also fixes a problem with the CMIS connector
> and another SolrJ-related issue (CONNECTORS-622 and CONNECTORS-623).
>
> This release candidate provides a better workaround for
> CONNECTORS-616 than RC5.  It also fixes CONNECTORS-617.
>
> This release candidate fixes one problem since RC4, which is
> the inconfigurability of the commit action path for Solr commits in
> the Solr connector.  This needed to be fixed to maintain backwards
> compatibility.  CONNECTORS-621.
>
> This release candidate fixes two problems since RC3.  The problems
> were in the included jars for the multiprocess example (CONNECTORS-619)
> and in connection leakage for JDBC handles (CONNECTORS-620).
>
> This release candidate fixes one problem since RC2.  The problem is
> CONNECTORS-618, which relates to MySQL performance.
>
> This release candidate fixes one additional problem since RC1.  The
> problem is CONNECTORS-616, and relates to Solr dropping connections
> during
> indexing.
>
> This release candidate fixes two other problems since RC0, both
> related to Solr 4.0.0 support.
> - CONNECTORS-613: The version of Tika used in Solr 4.0.0 cannot
> extract text unless told an accurate mime type.  While this is
> probably a Tika bug, in this ticket we at least make sure a good guess
> as to the mime type is sent to Solr.
> - CONNECTORS-614: Fix logic having to do with releasing idle Solr
> connections.  This shows up as socket timeout exceptions, because it
> becomes very easy to exhaust the Solr application server's thread pool
> when idle connections are not released in a timely way.
>
> This release includes a significant amount of long-planned upgrading
> and refactoring since Apache ManifoldCF 1.0.1, including:
> - Port to HttpComponents from commons-httpclient
> - Port to SolrJ from homegrown for the Solr connector, so that
> SolrCloud is supported
> - Improved NTLM support
> - Partial Kerberos support
> - Many other improvements, which are summarized in CHANGES.txt