You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by Guylaine BASSETTE <gu...@francelabs.com> on 2024/01/19 16:05:36 UTC
Solr ingestor for Solr 9
Hi Karl,
The Solr Ingestor needed a little fix to ingest Solr 9 documents. The
"httpmime" dependency was missing from the distribution packaging. With
Solr 9, a bug in Date field filling was discovered and fixed. We are
using it in our application.
We are currently using these fixes in our application.
I have created a Jira topic for this
(https://issues.apache.org/jira/projects/CONNECTORS/issues/CONNECTORS-1752)
and created the corresponding PR:
https://github.com/apache/manifoldcf/pull/161
Note that I have run "ant test" and all passed except for elasticsearch
IT-HSQLDB tests. I get back up until 2.25 and they were already showing
the same failures.
--
Fwd: Nouvelle signature
Best Regards,
Guylaine
France Labs – Your knowledge, now
Datafari Enterprise Search – Découvrez la version 5 / Discover our version 5
www.datafari.com <http://www.datafari.com>
Datafari Enterprise Search – Retrouvez-nous à WAICF Cannes
<https://www.worldaicannes.com/en> du 8 au 10 février
Logo_temp_waicf_2024 <https://www.worldaicannes.com/en>
Re: Solr ingestor for Solr 9
Posted by Guylaine BASSETTE <gu...@francelabs.com>.
Hi Mingchun,
Thanks for you share ! Indeed, I didn't wait all this time, because the
error I have with elasticsearch is the following:
run-IT-HSQLDB:
[junit] Testsuite:
org.apache.manifoldcf.agents.output.elasticsearch.tests.APISanityHSQLDBIT
[junit] Configuration file successfully read
[junit] [main] INFO org.eclipse.jetty.util.log - Logging
initialized @1888ms to org.eclipse.jetty.util.log.Slf4jLog
[junit] [main] INFO org.eclipse.jetty.server.Server -
jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git:
6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 11.0.21+9
[junit] [main] INFO org.eclipse.jetty.server.session -
DefaultSessionIdManager workerName=node0
[junit] [main] INFO org.eclipse.jetty.server.session - No
SessionScavenger set, using defaults
[junit] [main] INFO org.eclipse.jetty.server.session - node0
Scavenging every 660000ms
[junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler
- Started o.e.j.w.WebAppContext@56da52a7{ManifoldCF Crawler
Interface,/mcf-crawler-ui,file:///tmp/jetty-0_0_0_0-8346-mcf-crawler-ui_war-_mcf-crawler-ui-any-9370231220693628154/webapp/,AVAILABLE}{/home/guylaine/IdeaProjects/mon-manifoldcf/dist/web/war/mcf-crawler-ui.war}
[junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler
- Started o.e.j.w.WebAppContext@23ee75c5{ManifoldCF Authorities API
Webapp,/mcf-authority-service,file:///tmp/jetty-0_0_0_0-8346-mcf-authority-service_war-_mcf-authority-service-any-10372487271493735410/webapp/,AVAILABLE}{/home/guylaine/IdeaProjects/mon-manifoldcf/dist/web/war/mcf-authority-service.war}
[junit] ES working directory is
'/home/guylaine/IdeaProjects/mon-manifoldcf/connectors/elasticsearch/test-HSQLDB-output/.'
[junit] Unix process
[junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler
- Started o.e.j.w.WebAppContext@11a7ba62{ManifoldCF General API
Webapp,/mcf-api-service,file:///tmp/jetty-0_0_0_0-8346-mcf-api-service_war-_mcf-api-service-any-6486837083569953796/webapp/,AVAILABLE}{/home/guylaine/IdeaProjects/mon-manifoldcf/dist/web/war/mcf-api-service.war}
[junit] [main] INFO org.eclipse.jetty.server.AbstractConnector -
Started ServerConnector@4fe01803{HTTP/1.1, (http/1.1)}{0.0.0.0:8346}
[junit] [main] INFO org.eclipse.jetty.server.Server - Started @3884ms
[junit] ElasticSearch is starting...
[junit] Didn't reach ES; waiting...
[junit] Didn't reach ES; waiting...
[junit] Didn't reach ES; waiting...
It's a good news if you get through the tests successfully! Maybe I'm
lacking something in my environment...
Fwd: Nouvelle signature
Bien cordialement,
Guylaine
France Labs – Your knowledge, now
Datafari Enterprise Search – Découvrez la version 5 / Discover our version 5
www.datafari.com <http://www.datafari.com>
Datafari Enterprise Search – Retrouvez-nous à WAICF Cannes
<https://www.worldaicannes.com/en> du 8 au 10 février
Logo_temp_waicf_2024 <https://www.worldaicannes.com/en>
Le 28/01/2024 à 06:36, Mingchun Zhao a écrit :
> Hi Guylaine,
>
> Just FYI, allow me to share my test result with you for your PR:
> https://github.com/apache/manifoldcf/pull/161
>
> In my `ant test`, the result was OK, but it took 49 minutes. If an
> error occurs due to timeout during your test, it might be better to
> re-run your test.
>
> ```
> % ant clean clean-deps clean-core-deps make-core-deps make-deps build test
> ... ...
> test:
>
> BUILD SUCCESSFUL
> Total time: 49 minutes 2 seconds
> ```
>
> Regards,
> Mingchun
>
> 2024年1月20日(土) 1:06 Guylaine BASSETTE
> <gu...@francelabs.com>:
>
> Hi Karl,
>
> The Solr Ingestor needed a little fix to ingest Solr 9 documents.
> The "httpmime" dependency was missing from the distribution
> packaging. With Solr 9, a bug in Date field filling was discovered
> and fixed. We are using it in our application.
>
> We are currently using these fixes in our application.
>
> I have created a Jira topic for this
> (https://issues.apache.org/jira/projects/CONNECTORS/issues/CONNECTORS-1752)
> and created the corresponding PR:
> https://github.com/apache/manifoldcf/pull/161
>
> Note that I have run "ant test" and all passed except for
> elasticsearch IT-HSQLDB tests. I get back up until 2.25 and they
> were already showing the same failures.
>
> --
> Best Regards,
> Guylaine
>
> France Labs – Your knowledge, now
> Datafari Enterprise Search – Découvrez la version 5 / Discover our
> version 5
> www.datafari.com <http://www.datafari.com>
>
> Datafari Enterprise Search – Retrouvez-nous à WAICF Cannes
> <https://www.worldaicannes.com/en> du 8 au 10 février
>
> Logo_temp_waicf_2024 <https://www.worldaicannes.com/en>
>
Re: Solr ingestor for Solr 9
Posted by Mingchun Zhao <mi...@gmail.com>.
Hi Guylaine,
Just FYI, allow me to share my test result with you for your PR:
https://github.com/apache/manifoldcf/pull/161
In my `ant test`, the result was OK, but it took 49 minutes. If an error
occurs due to timeout during your test, it might be better to re-run your
test.
```
% ant clean clean-deps clean-core-deps make-core-deps make-deps build test
... ...
test:
BUILD SUCCESSFUL
Total time: 49 minutes 2 seconds
```
Regards,
Mingchun
2024年1月20日(土) 1:06 Guylaine BASSETTE <gu...@francelabs.com>:
> Hi Karl,
>
> The Solr Ingestor needed a little fix to ingest Solr 9 documents. The
> "httpmime" dependency was missing from the distribution packaging. With
> Solr 9, a bug in Date field filling was discovered and fixed. We are using
> it in our application.
>
> We are currently using these fixes in our application.
>
> I have created a Jira topic for this (
> https://issues.apache.org/jira/projects/CONNECTORS/issues/CONNECTORS-1752)
> and created the corresponding PR:
> https://github.com/apache/manifoldcf/pull/161
>
> Note that I have run "ant test" and all passed except for elasticsearch
> IT-HSQLDB tests. I get back up until 2.25 and they were already showing the
> same failures.
> --
> Best Regards,
> Guylaine
>
> France Labs – Your knowledge, now
> Datafari Enterprise Search – Découvrez la version 5 / Discover our version
> 5
> www.datafari.com
>
> Datafari Enterprise Search – Retrouvez-nous à WAICF Cannes
> <https://www.worldaicannes.com/en> du 8 au 10 février
>
> [image: Logo_temp_waicf_2024] <https://www.worldaicannes.com/en>
>
>
>