You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by Wolfinger Uwe <uw...@oegk.at> on 2021/07/23 11:12:36 UTC
JCIFS Connector File Size Attribute
Hi,
we are using the JCIFs shared drive connector to crawl windows shares. What we would like to have is, that the file size can be displayed in the search results, i.e. that an appropriate attribute is sent to solr.
According to this issue:
https://issues.apache.org/jira/projects/CONNECTORS/issues/CONNECTORS-1204
this should alfready work.
Unfortunately i am not able configure the corresponding job to send such an attribute. A look at
https://github.com/apache/manifoldcf/blob/trunk/connectors/jcifs/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedrive/SharedDriveConnector.java
shows, that only the following attributes are added as fields:
rd.addField("lastModified", lastModifiedDate.toString());
rd.addField("fileLastModified",DateParser.formatISO8601Date(lastModifiedDate));
rd.addField("createdOn", creationDate.toString());
rd.addField("fileCreatedOn",DateParser.formatISO8601Date(creationDate));
rd.addField("attributes", Integer.toString(attributes));
rd.addField("shareName", shareName);
am missing something? Or ist the fileSize attribute missing when populating the crawling result.
kind regards,
Uwe
AW: JCIFS Connector File Size Attribute
Posted by Wolfinger Uwe <uw...@oegk.at>.
Thank you very much fort he hint, now it works as expected!
Kind regards,
Uwe
-----Ursprüngliche Nachricht-----
Von: Karl Wright <da...@gmail.com>
Gesendet: Montag, 26. Juli 2021 18:28
An: dev <de...@manifoldcf.apache.org>
Betreff: Re: JCIFS Connector File Size Attribute
The parameter in the Solr connection UI is:
"Original size field name:"
Karl
On Mon, Jul 26, 2021 at 12:24 PM Wolfinger Uwe <uw...@oegk.at>
wrote:
> Do i need any extra configuration in the solr connection? When i look
> at the query string,that sends the request to solr, there is no field
> "originalSize".
>
> King Regards,
> Uwe
>
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: Karl Wright <da...@gmail.com>
> Gesendet: Freitag, 23. Juli 2021 20:34
> An: dev <de...@manifoldcf.apache.org>
> Betreff: Re: JCIFS Connector File Size Attribute
>
> Hi,
> The original size field is provided by the Repository Connector, and
> passed to the output connector.
>
> In this case, the code that sets the field is here:
>
> kawright@1USDKAWRIGHT:/mnt/c/wip/mcf/trunk$ grep -R
> "rd.setOriginalSize(originalLength);" . --include "*.java"
>
> ./connectors/jcifs/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedrive/SharedDriveConnector.jav
> : rd.setOriginalSize(originalLength);
>
> The code that uses this field and pushes it into Solr is configured in
> the Solr connection. That is probably why you are overlooking it.
>
> Thanks,
>
> Karl
>
>
> On Fri, Jul 23, 2021 at 10:13 AM Wolfinger Uwe <uw...@oegk.at>
> wrote:
>
> > Hi,
> >
> > we are using the JCIFs shared drive connector to crawl windows shares.
> > What we would like to have is, that the file size can be displayed
> > in the search results, i.e. that an appropriate attribute is sent to solr.
> >
> > According to this issue:
> > https://issues.apache.org/jira/projects/CONNECTORS/issues/CONNECTORS
> > -1
> > 204
> > this should alfready work.
> >
> > Unfortunately i am not able configure the corresponding job to send
> > such an attribute. A look at
> >
> >
> > https://github.com/apache/manifoldcf/blob/trunk/connectors/jcifs/con
> > ne
> > ctor/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedri
> > ve
> > /SharedDriveConnector.java
> >
> > shows, that only the following attributes are added as fields:
> > rd.addField("lastModified", lastModifiedDate.toString());
> >
> > rd.addField("fileLastModified",DateParser.formatISO8601Date(lastModi
> > fi edDate)); rd.addField("createdOn", creationDate.toString());
> > rd.addField("fileCreatedOn",DateParser.formatISO8601Date(creationDat
> > e) ); rd.addField("attributes", Integer.toString(attributes));
> > rd.addField("shareName", shareName);
> >
> > am missing something? Or ist the fileSize attribute missing when
> > populating the crawling result.
> >
> > kind regards,
> > Uwe
> >
> >
> >
>
Re: JCIFS Connector File Size Attribute
Posted by Karl Wright <da...@gmail.com>.
The parameter in the Solr connection UI is:
"Original size field name:"
Karl
On Mon, Jul 26, 2021 at 12:24 PM Wolfinger Uwe <uw...@oegk.at>
wrote:
> Do i need any extra configuration in the solr connection? When i look at
> the query string,that sends the request to solr, there is no field
> "originalSize".
>
> King Regards,
> Uwe
>
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: Karl Wright <da...@gmail.com>
> Gesendet: Freitag, 23. Juli 2021 20:34
> An: dev <de...@manifoldcf.apache.org>
> Betreff: Re: JCIFS Connector File Size Attribute
>
> Hi,
> The original size field is provided by the Repository Connector, and
> passed to the output connector.
>
> In this case, the code that sets the field is here:
>
> kawright@1USDKAWRIGHT:/mnt/c/wip/mcf/trunk$ grep -R
> "rd.setOriginalSize(originalLength);" . --include "*.java"
>
> ./connectors/jcifs/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedrive/SharedDriveConnector.jav
> : rd.setOriginalSize(originalLength);
>
> The code that uses this field and pushes it into Solr is configured in the
> Solr connection. That is probably why you are overlooking it.
>
> Thanks,
>
> Karl
>
>
> On Fri, Jul 23, 2021 at 10:13 AM Wolfinger Uwe <uw...@oegk.at>
> wrote:
>
> > Hi,
> >
> > we are using the JCIFs shared drive connector to crawl windows shares.
> > What we would like to have is, that the file size can be displayed in
> > the search results, i.e. that an appropriate attribute is sent to solr.
> >
> > According to this issue:
> > https://issues.apache.org/jira/projects/CONNECTORS/issues/CONNECTORS-1
> > 204
> > this should alfready work.
> >
> > Unfortunately i am not able configure the corresponding job to send
> > such an attribute. A look at
> >
> >
> > https://github.com/apache/manifoldcf/blob/trunk/connectors/jcifs/conne
> > ctor/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedrive
> > /SharedDriveConnector.java
> >
> > shows, that only the following attributes are added as fields:
> > rd.addField("lastModified", lastModifiedDate.toString());
> >
> > rd.addField("fileLastModified",DateParser.formatISO8601Date(lastModifi
> > edDate)); rd.addField("createdOn", creationDate.toString());
> > rd.addField("fileCreatedOn",DateParser.formatISO8601Date(creationDate)
> > ); rd.addField("attributes", Integer.toString(attributes));
> > rd.addField("shareName", shareName);
> >
> > am missing something? Or ist the fileSize attribute missing when
> > populating the crawling result.
> >
> > kind regards,
> > Uwe
> >
> >
> >
>
AW: JCIFS Connector File Size Attribute
Posted by Wolfinger Uwe <uw...@oegk.at>.
Do i need any extra configuration in the solr connection? When i look at the query string,that sends the request to solr, there is no field "originalSize".
King Regards,
Uwe
-----Ursprüngliche Nachricht-----
Von: Karl Wright <da...@gmail.com>
Gesendet: Freitag, 23. Juli 2021 20:34
An: dev <de...@manifoldcf.apache.org>
Betreff: Re: JCIFS Connector File Size Attribute
Hi,
The original size field is provided by the Repository Connector, and passed to the output connector.
In this case, the code that sets the field is here:
kawright@1USDKAWRIGHT:/mnt/c/wip/mcf/trunk$ grep -R "rd.setOriginalSize(originalLength);" . --include "*.java"
./connectors/jcifs/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedrive/SharedDriveConnector.jav
: rd.setOriginalSize(originalLength);
The code that uses this field and pushes it into Solr is configured in the Solr connection. That is probably why you are overlooking it.
Thanks,
Karl
On Fri, Jul 23, 2021 at 10:13 AM Wolfinger Uwe <uw...@oegk.at>
wrote:
> Hi,
>
> we are using the JCIFs shared drive connector to crawl windows shares.
> What we would like to have is, that the file size can be displayed in
> the search results, i.e. that an appropriate attribute is sent to solr.
>
> According to this issue:
> https://issues.apache.org/jira/projects/CONNECTORS/issues/CONNECTORS-1
> 204
> this should alfready work.
>
> Unfortunately i am not able configure the corresponding job to send
> such an attribute. A look at
>
>
> https://github.com/apache/manifoldcf/blob/trunk/connectors/jcifs/conne
> ctor/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedrive
> /SharedDriveConnector.java
>
> shows, that only the following attributes are added as fields:
> rd.addField("lastModified", lastModifiedDate.toString());
>
> rd.addField("fileLastModified",DateParser.formatISO8601Date(lastModifi
> edDate)); rd.addField("createdOn", creationDate.toString());
> rd.addField("fileCreatedOn",DateParser.formatISO8601Date(creationDate)
> ); rd.addField("attributes", Integer.toString(attributes));
> rd.addField("shareName", shareName);
>
> am missing something? Or ist the fileSize attribute missing when
> populating the crawling result.
>
> kind regards,
> Uwe
>
>
>
Re: JCIFS Connector File Size Attribute
Posted by Karl Wright <da...@gmail.com>.
Hi,
The original size field is provided by the Repository Connector, and passed
to the output connector.
In this case, the code that sets the field is here:
kawright@1USDKAWRIGHT:/mnt/c/wip/mcf/trunk$ grep -R
"rd.setOriginalSize(originalLength);" . --include "*.java"
./connectors/jcifs/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedrive/SharedDriveConnector.jav
: rd.setOriginalSize(originalLength);
The code that uses this field and pushes it into Solr is configured in the
Solr connection. That is probably why you are overlooking it.
Thanks,
Karl
On Fri, Jul 23, 2021 at 10:13 AM Wolfinger Uwe <uw...@oegk.at>
wrote:
> Hi,
>
> we are using the JCIFs shared drive connector to crawl windows shares.
> What we would like to have is, that the file size can be displayed in the
> search results, i.e. that an appropriate attribute is sent to solr.
>
> According to this issue:
> https://issues.apache.org/jira/projects/CONNECTORS/issues/CONNECTORS-1204
> this should alfready work.
>
> Unfortunately i am not able configure the corresponding job to send such
> an attribute. A look at
>
>
> https://github.com/apache/manifoldcf/blob/trunk/connectors/jcifs/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedrive/SharedDriveConnector.java
>
> shows, that only the following attributes are added as fields:
> rd.addField("lastModified", lastModifiedDate.toString());
>
> rd.addField("fileLastModified",DateParser.formatISO8601Date(lastModifiedDate));
> rd.addField("createdOn", creationDate.toString());
> rd.addField("fileCreatedOn",DateParser.formatISO8601Date(creationDate));
> rd.addField("attributes", Integer.toString(attributes));
> rd.addField("shareName", shareName);
>
> am missing something? Or ist the fileSize attribute missing when
> populating the crawling result.
>
> kind regards,
> Uwe
>
>
>