You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by Karl Wright <da...@gmail.com> on 2019/09/26 01:21:41 UTC
[CANCEL][VOTE] Release Apache ManifoldCF 2.14, RC1
Vote canceled to address the duplicate jars Cihad found.
Karl
On Wed, Sep 25, 2019 at 8:53 PM Karl Wright <da...@gmail.com> wrote:
> It's the generic connector that sucks this in. I'm going to disable it
> and see if everything still works or not.
>
> Karl
>
>
> On Wed, Sep 25, 2019 at 8:42 PM Karl Wright <da...@gmail.com> wrote:
>
>> I looked at the JSP exception.
>>
>> Velocity 1.7 (which is what we include) depends on Commons Collections
>> 3.2.1. We include version 3.2.2, which should be fine, and it indeed has
>> that class:
>>
>> >>>>>>
>> C:\wip\mcf\trunk\lib>dir commons-collections-3.2.2.jar
>> Volume in drive C is Windows
>> Volume Serial Number is F4D8-E4E0
>>
>> Directory of C:\wip\mcf\trunk\lib
>>
>> 09/06/2019 04:03 PM 588,337 commons-collections-3.2.2.jar
>> 1 File(s) 588,337 bytes
>> 0 Dir(s) 179,446,546,432 bytes free
>>
>> C:\wip\mcf\trunk\lib>tar tf commons-collections-3.2.2.jar | grep
>> ExtendedProperties
>>
>> org/apache/commons/collections/ExtendedProperties$PropertiesTokenizer.class
>> org/apache/commons/collections/ExtendedProperties$1.class
>> org/apache/commons/collections/ExtendedProperties$PropertiesReader.class
>> org/apache/commons/collections/ExtendedProperties.class
>>
>> C:\wip\mcf\trunk\lib>
>> <<<<<<
>>
>> The only other question is whether it's in the right place. I noted that
>> the velocity jar and the commons-collections jar appear BOTH at the root
>> level AND in connector-lib. This doesn't seem right; if it's at the root
>> level it should not be in connector-lib too. Let me see if I can figure
>> out why that is there.
>>
>> Karl
>>
>>
>> On Wed, Sep 25, 2019 at 8:20 PM Karl Wright <da...@gmail.com> wrote:
>>
>>> How was this JSON generated? By hand? Or exported? If exported, by
>>> what version of MCF?
>>>
>>> I ask because there was an issue that was fixed back around 2.10 or so
>>> (if memory serves) having to do with the export of JSON. The JSON format
>>> previously emitted did not preserve order in some cases; after the change,
>>> it did. Might be related to API usage in this case; can't be sure.
>>>
>>> At any rate, the problem is coming from viewing a transformation step in
>>> your pipeline. What transformation connections are you using?
>>>
>>> Karl
>>>
>>>
>>>
>>>
>>> On Wed, Sep 25, 2019 at 7:01 PM Cihad Guzel <cg...@gmail.com> wrote:
>>>
>>>> Hi Karl,
>>>>
>>>> Shortly;
>>>> 1- I have created a job via ManifoldCF API. There is not any problem,
>>>> no error, all is well.
>>>> 2- Then I try to open the Edit Job page from UI and UI won't display it
>>>> properly. There is the 500 error on the popup.
>>>>
>>>> I didn't move my current installation. I have downloaded new ManifoldCF
>>>> version and have run in a clean new environment. Then, I have sent the JSON
>>>> object to my new ManifoldCF API.
>>>>
>>>> I agree with you on this:
>>>> "The actual error you are getting is because of a JSP exception. The
>>>> JSP exception is not being handled properly, possibly because of a version
>>>> mismatch between the included version of Velocity and Apache Commons
>>>> Collections. "
>>>>
>>>> Regards,
>>>> Cihad Guzel
>>>>
>>>> Karl Wright <da...@gmail.com>, 26 Eyl 2019 Per, 01:31 tarihinde
>>>> şunu yazdı:
>>>>
>>>>> The actual error you are getting is because of a JSP exception. The
>>>>> JSP exception is not being handled properly, possibly because of a version
>>>>> mismatch between the included version of Velocity and Apache Commons
>>>>> Collections. But if that is real, it's odd because it's occurring when the
>>>>> velocity context is being initialized!! That means it should be happening
>>>>> on every page load in the UI, and it most certainly does not -- not for me,
>>>>> not for anyone who ran the tests. So I suspect a setup problem of some
>>>>> kind, especially if you're trying to move your 2.12 installation to 2.14 in
>>>>> place. Lots of jars moved around and you might have a jar conflict.
>>>>>
>>>>> Karl
>>>>>
>>>>> On Wed, Sep 25, 2019 at 6:25 PM Karl Wright <da...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Cihad,
>>>>>> Using the API should not ever invoke JSP pages, so I do not think you
>>>>>> are actually communicating with the API. Unless what you are saying is
>>>>>> that you create the job, and then the UI won't display it properly?
>>>>>>
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>> On Wed, Sep 25, 2019 at 6:11 PM Cihad Guzel <cg...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Firstly, Thanks @Furkan, all tests are passed with Java8 now.
>>>>>>>
>>>>>>> On the other hand, I test the ManifoldCF API. There are some errors.
>>>>>>> I have created a job using ManifoldCF API successfully (versions: 2.12,
>>>>>>> 2.13, 2.14). Then I have HTTP 500 error if I click
>>>>>>> "mcf-crawler-ui/editjob.jsp" . I get this error when only I use
>>>>>>> Manifoldcf-2.13 and Manifoldcf-2.14. I don't have any error when I use
>>>>>>> Manifoldcf-2.12.
>>>>>>>
>>>>>>> I sent following JSON object to the manifoldCF API to create new
>>>>>>> job.
>>>>>>>
>>>>>>> {"job":{"children":[{"type":"start_mode","value":"manual"},{"type":"hopcount_mode","value":"accurate"},{"type":"run_mode","value":"continuous"},{"type":"recrawl_interval","value":"86400000"},{"type":"reseed_interval","value":"3600000"},{"type":"description","value":"file_filecrawler2"},{"type":"repository_connection","value":"file_filecrawler2"},{"type":"priority","value":"1"},{"type":"pipelinestage","children":[{"type":"stage_id","value":"0"},{"type":"stage_isoutput","value":"false"},{"type":"stage_connectionname","value":"Allowed Documents"},{"type":"stage_specification","children":[{"type":"extensions","value":"xlsx\r\nrtf\r\ncsv\r\ndocx\r\npptx\r\ntxt\r\npdf\r\nppt\r\nxml\r\ndoc\r\nodp\r\nxls\r\nodt\r\nods"},{"type":"mimetypes","value":"application\/xml\r\napplication\/msword\r\napplication\/pdf\r\napplication\/vnd.ms-powerpoint\r\napplication\/vnd.openxmlformats-officedocument.wordprocessingml.document\r\napplication\/rtf\r\napplication\/vnd.oasis.opendocument.text\r\napplication\/vnd.openxmlformats-officedocument.presentationml.presentation\r\napplication\/vnd.openxmlformats-officedocument.spreadsheetml.sheet\r\ntext\/csv\r\napplication\/vnd.oasis.opendocument.presentation\r\napplication\/vnd.ms-excel\r\napplication\/vnd.oasis.opendocument.spreadsheet\r\ntext\/plain"},{"type":"mindate","value":"","attribute_value":"36593"},{"_type":"minlength","value":"","attribute_value":"1"},{"_type":"maxlength","value":"","attribute_value":"100000000"}]}]},{"_type":"pipelinestage","children":[{"type":"stage_id","value":"1"},{"type":"stage_isoutput","value":"false"},{"type":"stage_connectionname","value":"Source Type"},{"type":"stage_prerequisite","value":"0"},{"type":"stage_specification","children":[{"type":"expression","attribute_parameter":"source_type","_attribute_value":"Windows Shares"},{"_type":"expression","attribute_parameter":"crawler_id","_attribute_value":"file_filecrawler2"},{"_type":"keepAllMetadata","value":"","attribute_value":"true"},{"_type":"filterEmpty","value":"","attribute_value":"true"}]}]},{"_type":"pipelinestage","children":[{"type":"stage_id","value":"2"},{"type":"stage_isoutput","value":"true"},{"type":"stage_connectionname","value":"Solr"},{"type":"stage_prerequisite","value":"1"}]},{"type":"document_specification","children":[{"type":"security","value":"","attribute_value":"off"},{"_type":"sharesecurity","value":"","attribute_value":"off"},{"_type":"parentfoldersecurity","value":"","attribute_value":"off"},{"_type":"startpoint","include":[{"attribute_indexable":"yes","_attribute_filespec":"*","_value":"","attribute_type":"file"},{"_attribute_filespec":"*","_value":"","attribute_type":"directory"}],"_attribute_path":"SharedFiles","_value":""}]}]}}
>>>>>>>
>>>>>>>
>>>>>>> The error message as follow:
>>>>>>> HTTP ERROR 500
>>>>>>>
>>>>>>> Problem accessing /mcf-crawler-ui/editjob.jsp. Reason:
>>>>>>>
>>>>>>> Server Error
>>>>>>>
>>>>>>> Caused by:
>>>>>>>
>>>>>>> org.apache.jasper.JasperException: An exception occurred processing JSP page /editjob.jsp at line 716
>>>>>>>
>>>>>>> 713: {
>>>>>>> 714: try
>>>>>>> 715: {
>>>>>>> 716: transformationConnector.outputSpecificationHeader(new org.apache.manifoldcf.ui.jsp.JspWrapper(out,adminprofile),pageContext.getRequest().getLocale(),pipelineSpecifications[j],1+j,tabsArray);
>>>>>>> 717: }
>>>>>>> 718: finally
>>>>>>> 719: {
>>>>>>>
>>>>>>>
>>>>>>> Stacktrace:
>>>>>>> at org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:521)
>>>>>>> at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:412)
>>>>>>> at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
>>>>>>> at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
>>>>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>>>>>>> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)
>>>>>>> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>>>>>>> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>>>>>>> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>>>>>>> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>>>>>>> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
>>>>>>> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>>>>>>> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>>>>>>> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
>>>>>>> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>>>>>>> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>>>>>>> at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
>>>>>>> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>>>>>>> at org.eclipse.jetty.server.Server.handle(Server.java:497)
>>>>>>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
>>>>>>> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)
>>>>>>> at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>>>>>>> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)
>>>>>>> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)
>>>>>>> at java.lang.Thread.run(Thread.java:748)
>>>>>>> Caused by: javax.servlet.ServletException: java.lang.NoClassDefFoundError: org/apache/commons/collections/ExtendedProperties
>>>>>>> at org.apache.jasper.runtime.PageContextImpl.doHandlePageException(PageContextImpl.java:865)
>>>>>>> at org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContextImpl.java:794)
>>>>>>> at org.apache.jsp.editjob_jsp._jspService(editjob_jsp.java:3113)
>>>>>>> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
>>>>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>>>>>>> at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:388)
>>>>>>> ... 23 more
>>>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/commons/collections/ExtendedProperties
>>>>>>> at org.apache.velocity.runtime.RuntimeInstance.<init>(RuntimeInstance.java:183)
>>>>>>> at org.apache.velocity.app.VelocityEngine.<init>(VelocityEngine.java:60)
>>>>>>> at org.apache.manifoldcf.core.i18n.Messages.createVelocityEngine(Messages.java:55)
>>>>>>> at org.apache.manifoldcf.ui.i18n.Messages.outputResourceWithVelocity(Messages.java:132)
>>>>>>> at org.apache.manifoldcf.agents.transformation.documentfilter.Messages.outputResourceWithVelocity(Messages.java:136)
>>>>>>> at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.outputSpecificationHeader(DocumentFilter.java:288)
>>>>>>> at org.apache.jsp.editjob_jsp._jspService(editjob_jsp.java:953)
>>>>>>> ... 26 more
>>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.commons.collections.ExtendedProperties
>>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>> ... 33 more
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Cihad Guzel
>>>>>>>
>>>>>>>
>>>>>>> Furkan KAMACI <fu...@gmail.com>, 25 Eyl 2019 Çar, 15:14
>>>>>>> tarihinde şunu yazdı:
>>>>>>>
>>>>>>>> Hi Cihad,
>>>>>>>>
>>>>>>>> That package is removed at Java 11. It is possible that you
>>>>>>>> compiled the
>>>>>>>> project via Java 11 which is not supported at ManifoldCF yet.
>>>>>>>>
>>>>>>>> Kind Regards,
>>>>>>>> Furkan KAMACI
>>>>>>>>
>>>>>>>> 25 Eyl 2019 Çar, saat 15:00 tarihinde Cihad Guzel <
>>>>>>>> cguzelg@gmail.com> şunu
>>>>>>>> yazdı:
>>>>>>>>
>>>>>>>> > Hi Karl,
>>>>>>>> >
>>>>>>>> > I have following errors:
>>>>>>>> >
>>>>>>>> > [INFO] -------------< org.apache.manifoldcf:mcf-connector-common
>>>>>>>> > >-------------
>>>>>>>> > [INFO] Building ManifoldCF - Connector-Common 2.14
>>>>>>>> > [5/63]
>>>>>>>> > [INFO] --------------------------------[ jar
>>>>>>>> > ]---------------------------------
>>>>>>>> > [INFO]
>>>>>>>> > [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @
>>>>>>>> > mcf-connector-common ---
>>>>>>>> > [INFO]
>>>>>>>> > [INFO] --- maven-remote-resources-plugin:1.5:process (default) @
>>>>>>>> > mcf-connector-common ---
>>>>>>>> > [INFO]
>>>>>>>> > [INFO] --- maven-resources-plugin:2.5:resources
>>>>>>>> (default-resources) @
>>>>>>>> > mcf-connector-common ---
>>>>>>>> > [debug] execute contextualize
>>>>>>>> > [INFO] Using 'UTF-8' encoding to copy filtered resources.
>>>>>>>> > [INFO] Copying 1 resource
>>>>>>>> > [INFO] Copying 3 resources
>>>>>>>> > [INFO]
>>>>>>>> > [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @
>>>>>>>> > mcf-connector-common ---
>>>>>>>> > [INFO] Compiling 59 source files to
>>>>>>>> >
>>>>>>>> >
>>>>>>>> /Users/cguzel/Projects/apache/svn/mcf-tag-2.14-RC1/framework/connector-common/target/classes
>>>>>>>> > [INFO]
>>>>>>>> -------------------------------------------------------------
>>>>>>>> > [ERROR] COMPILATION ERROR :
>>>>>>>> > [INFO]
>>>>>>>> -------------------------------------------------------------
>>>>>>>> > [ERROR]
>>>>>>>> >
>>>>>>>> >
>>>>>>>> /Users/cguzel/Projects/apache/svn/mcf-tag-2.14-RC1/framework/connector-common/src/main/java/org/apache/manifoldcf/connectorcommon/common/CommonsHTTPSender.java:[60,21]
>>>>>>>> > error: package javax.xml.soap does not exist
>>>>>>>> > [ERROR]
>>>>>>>> >
>>>>>>>> >
>>>>>>>> /Users/cguzel/Projects/apache/svn/mcf-tag-2.14-RC1/framework/connector-common/src/main/java/org/apache/manifoldcf/connectorcommon/common/CommonsHTTPSender.java:[61,21]
>>>>>>>> > error: package javax.xml.soap does not exist
>>>>>>>> > [ERROR]
>>>>>>>> >
>>>>>>>> >
>>>>>>>> /Users/cguzel/Projects/apache/svn/mcf-tag-2.14-RC1/framework/connector-common/src/main/java/org/apache/manifoldcf/connectorcommon/common/CommonsHTTPSender.java:[62,21]
>>>>>>>> > error: package javax.xml.soap does not exist
>>>>>>>> > [INFO] 3 errors
>>>>>>>> >
>>>>>>>> > Regards,
>>>>>>>> > Cihad Guzel
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > Karl Wright <da...@gmail.com>, 25 Eyl 2019 Çar, 02:01
>>>>>>>> tarihinde şunu
>>>>>>>> > yazdı:
>>>>>>>> >
>>>>>>>> > > Ran all tests.
>>>>>>>> > >
>>>>>>>> > > +1 from me.
>>>>>>>> > >
>>>>>>>> > > Karl
>>>>>>>> > >
>>>>>>>> > >
>>>>>>>> > > On Tue, Sep 24, 2019 at 1:29 PM Karl Wright <da...@gmail.com>
>>>>>>>> wrote:
>>>>>>>> > >
>>>>>>>> > > > Please vote on whether to release Apache ManifoldCF 2.14, RC0.
>>>>>>>> > > >
>>>>>>>> > > > There is a release tag at
>>>>>>>> > > >
>>>>>>>> https://svn.apache.org/repos/asf/manifoldcf/tags/release-2.14-RC1 .
>>>>>>>> > > > There is a release artifact at
>>>>>>>> > > >
>>>>>>>> >
>>>>>>>> https://dist.apache.org/repos/dist/dev/manifoldcf/apache-manifoldcf-2.14
>>>>>>>> > > .
>>>>>>>> > > >
>>>>>>>> > > > Note that this release of ManifoldCF has a new connector --
>>>>>>>> the Content
>>>>>>>> > > > Services Web Services connector. In order to integrate this
>>>>>>>> connector,
>>>>>>>> > > > changes to the dependencies for web services had to be made,
>>>>>>>> so I urge
>>>>>>>> > > > those using the Alfresco and CMIS connectors to verify this
>>>>>>>> release if
>>>>>>>> > at
>>>>>>>> > > > all possible. Integration tests pass but there's nothing
>>>>>>>> like an
>>>>>>>> > on-site
>>>>>>>> > > > test.
>>>>>>>> > > >
>>>>>>>> > > > RC1 also contains a fix for CONNECTORS-1623.
>>>>>>>> > > >
>>>>>>>> > > > Thanks,
>>>>>>>> > > > Karl
>>>>>>>> > > >
>>>>>>>> > >
>>>>>>>> >
>>>>>>>>
>>>>>>>