You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@manifoldcf.apache.org by "Hotchkiss, David" <dh...@iso-ne.com> on 2017/10/31 15:57:48 UTC

FileOutputConnector.removeDocument() root level path check.

Using ManifoldCF v 2.8.1: I think there is a problem with the Filesystem
Output Connector's removeDocument method.

During testing, we noticed that documents removed from the source
repository did not result in a zero-length file in the file system
destination.

When the "removeDocument()" method checks for existence of the root, it
constructs a "File" object from an empty StringBuffer instead of the
root path for the output destination.  This causes removeDocument to
return on condition that the path does not exist.

Is this the expected behavior?

Thanks,

  //david

Re: [EXT] Re: FileOutputConnector.removeDocument() root level path check.

Posted by Karl Wright <da...@gmail.com>.

It's not quite that simple.

Have a look at addOrReplaceDocumentWithException().  The following code
computes the file name given the URI:

>>>>>>
    // Establish a session
    getSession();

    FileOutputConfig config = getConfigParameters(null);

    FileOutputSpecs specs = new
FileOutputSpecs(getSpecNode(outputDescription.getSpecification()));;
    StringBuffer path = new StringBuffer();

    String errorCode = "OK";
    String errorDesc = null;

    try {
      /*
        * make file path
        */
      if (specs.getRootPath() != null) {
        path.append(specs.getRootPath());
      }

      // If the path does not yet exist at the root level, it is dangerous
to create it.
      File currentPath = new File(path.toString());
      if (!currentPath.exists())
        throw new ManifoldCFException("Root path does not yet exist:
'"+currentPath+"'");
      if (!currentPath.isDirectory())
        throw new ManifoldCFException("Root path is not a directory:
'"+currentPath+"'");

      String filePath = documentURItoFilePath(documentURI);

<<<<<<

You will want to adapt this code so that the file name is computed in the
same way for the removeDocument() method.

Thanks,
Karl


On Tue, Oct 31, 2017 at 1:20 PM, Hotchkiss, David <dh...@iso-ne.com>
wrote:

> Hi Karl,
>
> I'm happy to submit a patch with the issue.
>
> I have not read deeply into the framework API's, but I tested a simple
> "hack" to pass to the StringBuffer constructor and it worked:
>
>   StringBuffer path = new StringBuffer(outputDescription.substring(0,
> outputDescription.length() - 1));
>
> Obviously, the patch should include precondition checks on the
> outputDescription length etc.
>
> Is this an appropriate use of the "outputDescription" argument to
> removeDocument() or can you suggest more reliable way to obtain the root
> directory? (e.g. derived some how  from the Job or FileOutputConfig)
>
> Thanks,
>
>   //david
>
> On Tue, 2017-10-31 at 12:55 -0400, Karl Wright wrote:
>
>
> *** EXTERNAL email. Please be cautious and evaluate before you click on
> links, open attachments, or provide credentials. ***
>
>
> Hi David,
>
>
>
> This is not the expected behavior.  The removeDocument() method should be
> looking for the existence of the root directory, and returning if that
> doesn't exist, before attempting to remove the specified document.
>
>
>
> Please create a ticket, and if you are interested, create a patch.
> Thanks!
>
>
>
> Karl
>
>
>
>
> On Tue, Oct 31, 2017 at 11:57 AM, Hotchkiss, David <dh...@iso-ne.com>
> wrote:
>
> Using ManifoldCF v 2.8.1: I think there is a problem with the Filesystem
> Output Connector's removeDocument method.
>
> During testing, we noticed that documents removed from the source
> repository did not result in a zero-length file in the file system
> destination.
>
> When the "removeDocument()" method checks for existence of the root, it
> constructs a "File" object from an empty StringBuffer instead of the
> root path for the output destination.  This causes removeDocument to
> return on condition that the path does not exist.
>
> Is this the expected behavior?
>
> Thanks,
>
>   //david
>
>
>
>
>

Re: [EXT] Re: FileOutputConnector.removeDocument() root level path check.

Posted by "Hotchkiss, David" <dh...@iso-ne.com>.

Hi Karl,

I'm happy to submit a patch with the issue.

I have not read deeply into the framework API's, but I tested a simple "hack" to pass to the StringBuffer constructor and it worked:

StringBuffer path = new StringBuffer(outputDescription.substring(0, outputDescription.length() - 1));

Obviously, the patch should include precondition checks on the outputDescription length etc.

Is this an appropriate use of the "outputDescription" argument to removeDocument() or can you suggest more reliable way to obtain the root directory? (e.g. derived some how from the Job or FileOutputConfig)

Thanks,

//david

On Tue, 2017-10-31 at 12:55 -0400, Karl Wright wrote:

*** EXTERNAL email. Please be cautious and evaluate before you click on links, open attachments, or provide credentials. ***

Hi David,

This is not the expected behavior. The removeDocument() method should be looking for the existence of the root directory, and returning if that doesn't exist, before attempting to remove the specified document.

Please create a ticket, and if you are interested, create a patch. Thanks!

Karl

On Tue, Oct 31, 2017 at 11:57 AM, Hotchkiss, David <dh...@iso-ne.com>> wrote:
Using ManifoldCF v 2.8.1: I think there is a problem with the Filesystem
Output Connector's removeDocument method.

During testing, we noticed that documents removed from the source
repository did not result in a zero-length file in the file system
destination.

When the "removeDocument()" method checks for existence of the root, it
constructs a "File" object from an empty StringBuffer instead of the
root path for the output destination. This causes removeDocument to
return on condition that the path does not exist.

Is this the expected behavior?

Thanks,

//david

Re: FileOutputConnector.removeDocument() root level path check.

Posted by Karl Wright <da...@gmail.com>.

Hi David,

This is not the expected behavior.  The removeDocument() method should be
looking for the existence of the root directory, and returning if that
doesn't exist, before attempting to remove the specified document.

Please create a ticket, and if you are interested, create a patch.  Thanks!

Karl

On Tue, Oct 31, 2017 at 11:57 AM, Hotchkiss, David <dh...@iso-ne.com>
wrote:

> Using ManifoldCF v 2.8.1: I think there is a problem with the Filesystem
> Output Connector's removeDocument method.
>
> During testing, we noticed that documents removed from the source
> repository did not result in a zero-length file in the file system
> destination.
>
> When the "removeDocument()" method checks for existence of the root, it
> constructs a "File" object from an empty StringBuffer instead of the
> root path for the output destination.  This causes removeDocument to
> return on condition that the path does not exist.
>
> Is this the expected behavior?
>
> Thanks,
>
>   //david
>