You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by "Rohit Batta (Jira)" <ji...@apache.org> on 2020/02/14 17:19:00 UTC
[jira] [Comment Edited] (CONNECTORS-1636) ElasticSearch Connector
not working with ingest pipeline processor attachment
[ https://issues.apache.org/jira/browse/CONNECTORS-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17037123#comment-17037123 ]
Rohit Batta edited comment on CONNECTORS-1636 at 2/14/20 5:18 PM:
------------------------------------------------------------------
Hi Karl,
I have fixed it by validating pipeline and processing base64 in case of pipeline uses attachment processor.
I am facing one issue now if I add some other fields to form by declaring those to `ElasticSearchParam.java`, the form is not rendered.
{code:html}
<tr>
<td class="description"><nobr>$Encoder.bodyEscape($ResourceBundle.getString('ElasticSearchConnector.ReferenceIndexNameColon'))</nobr></td>
<td class="value"><input name="referenceindexname" type="text" value="$Encoder.attributeEscape($REFERENCEINDEXNAME)" size="24" /></td>
</tr>
{code}
Error logs prints
{code:none}
// Some comments here
org.apache.velocity.exception.MethodInvocationException: Invocation of method 'attributeEscape' in class java.lang.Class threw exception java.lang.NullPointerException at editConfiguration_Parameters.html[line 103, column 64]
at org.apache.velocity.runtime.parser.node.ASTMethod.handleInvocationException(ASTMethod.java:243)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:187)
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:280)
at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:369)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342)
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:106)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342)
at org.apache.velocity.Template.merge(Template.java:356)
at org.apache.velocity.Template.merge(Template.java:260)
at org.apache.velocity.app.VelocityEngine.mergeTemplate(VelocityEngine.java:354)
at org.apache.manifoldcf.ui.i18n.Messages.outputResourceWithVelocity(Messages.java:159)
at org.apache.manifoldcf.agents.output.elasticsearch.Messages.outputResourceWithVelocity(Messages.java:136)
at org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnector.outputResource(ElasticSearchConnector.java:269)
at org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnector.outputConfigurationBody(ElasticSearchConnector.java:294)
at org.apache.manifoldcf.core.interfaces.ConnectorFactory.outputThisConfigurationBody(ConnectorFactory.java:83)
at org.apache.manifoldcf.agents.interfaces.OutputConnectorFactory.outputConfigurationBody(OutputConnectorFactory.java:107)
at org.apache.jsp.editoutput_jsp._jspService(editoutput_jsp.java:694)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:388)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at org.apache.manifoldcf.ui.util.Encoder.attributeEscape(Encoder.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:395)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:384)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:173)
... 42 more
{code}
I have validated the attribute name and configuration class too.
was (Author: rohitbatta):
Hi Karl,
I have fixed it by validating pipeline and processing base64 in case of pipeline uses attachment processor.
I am facing one issue now if I add some other fields to form by declaring those to `ElasticSearchParam.java`, the form is not rendered.
{code:html}
<tr>
<td class="description"><nobr>$Encoder.bodyEscape($ResourceBundle.getString('ElasticSearchConnector.ReferenceIndexNameColon'))</nobr></td>
<td class="value"><input name="referenceindexname" type="text" value="$Encoder.attributeEscape($REFERENCEINDEXNAME)" size="24" /></td>
</tr>
{code}
Error logs prints
{code:txt}
// Some comments here
org.apache.velocity.exception.MethodInvocationException: Invocation of method 'attributeEscape' in class java.lang.Class threw exception java.lang.NullPointerException at editConfiguration_Parameters.html[line 103, column 64]
at org.apache.velocity.runtime.parser.node.ASTMethod.handleInvocationException(ASTMethod.java:243)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:187)
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:280)
at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:369)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342)
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:106)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342)
at org.apache.velocity.Template.merge(Template.java:356)
at org.apache.velocity.Template.merge(Template.java:260)
at org.apache.velocity.app.VelocityEngine.mergeTemplate(VelocityEngine.java:354)
at org.apache.manifoldcf.ui.i18n.Messages.outputResourceWithVelocity(Messages.java:159)
at org.apache.manifoldcf.agents.output.elasticsearch.Messages.outputResourceWithVelocity(Messages.java:136)
at org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnector.outputResource(ElasticSearchConnector.java:269)
at org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnector.outputConfigurationBody(ElasticSearchConnector.java:294)
at org.apache.manifoldcf.core.interfaces.ConnectorFactory.outputThisConfigurationBody(ConnectorFactory.java:83)
at org.apache.manifoldcf.agents.interfaces.OutputConnectorFactory.outputConfigurationBody(OutputConnectorFactory.java:107)
at org.apache.jsp.editoutput_jsp._jspService(editoutput_jsp.java:694)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:388)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at org.apache.manifoldcf.ui.util.Encoder.attributeEscape(Encoder.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:395)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:384)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:173)
... 42 more
{code}
I have validated the attribute name and configuration class too.
> ElasticSearch Connector not working with ingest pipeline processor attachment
> -----------------------------------------------------------------------------
>
> Key: CONNECTORS-1636
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1636
> Project: ManifoldCF
> Issue Type: Bug
> Components: Elastic Search connector
> Affects Versions: ManifoldCF 2.15
> Reporter: Rohit Batta
> Assignee: Karl Wright
> Priority: Major
> Labels: ManifoldCF, connector, elasticsearch, manifoldcf
> Fix For: ManifoldCF next
>
>
> While using Apache manifoldcf elasticsearch connector for elasticsearch version 6.6.x, I found that connector is not working as expected for pipeline processor "attachment".
> The processor requires Base64 String to process input stream to content.
> It is working for "mapper-attachment" plugin but that plugin is deprecated in newer versions of elasticsearch.
> In case elasticsearch pipeline is used and mapper-attachment is set to false. then the content is processed as byte Array to index document, which is not correct type for indexing to elasticsearch.
>
> {code:java}
> if (!useMapperAttachments && inputStream != null) {
> if (contentAttributeName != null) {
> Reader r = new InputStreamReader(inputStream, Consts.UTF_8);
> if (needComma) {
> pw.print(",");
> }
> pw.append(jsonStringEscape(contentAttributeName)).append(" : \"");
> char[] buffer = new char[65536];
> while (true) {
> int amt = r.read(buffer, 0, buffer.length);
> if (amt == -1)
> break;
> for (int j = 0; j < amt; j++) {
> final char x = buffer[j];
> if (x == '\n')
> pw.append('\\').append('n');
> else if (x == '\r')
> pw.append('\\').append('r');
> else if (x == '\t')
> pw.append('\\').append('t');
> else if (x == '\b')
> pw.append('\\').append('b');
> else if (x == '\f')
> pw.append('\\').append('f');
> else if (x < 32) {
> pw.append("\\u").append(String.format(Locale.ROOT, "%04x", (int) x));
> } else {
> if (x == '\"' || x == '\\' || x == '/')
> pw.append('\\');
> pw.append(x);
> }
> }
> }
> pw.append("\"");
> needComma = true;
> }
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)