You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by David Arroyo <ar...@gmail.com> on 2017/02/21 23:27:04 UTC
[ CONNECTORS-1290 ] Nuxeo connector for Apache ManifoldCF
Hello everyone
I'm updating the Nuxeo connector with the nuxeo java client. I have found
an issue with the libraries jackson-annotations and jackson-databind due to
monifoldcf uses versions 2.1.1 and 2.1.3 respectively, and the nuxeo java
client needs at least 2.6.0 version for both.
If i compile and build the last manifold with the nexus connector, it works
fine, but I have not tested it with other connectors. Is there any problem
for update the libraries to 2.6.0 version?
Moreover, I am adding new functionality to connector for filter the type of
files we want to index.
Aside of that, nuxeo java client seems to have a bug for recovery the
attachment content, so it can not be recovered. Therefore, I think there
are 3 possible options:
1. Wait the issue be fixed [1]
2. Generate a patch for the nuxeo client (Not sure how to share this
patched lib)
3. Not included the content of the attachments
Regards.
[1] https://github.com/nuxeo/nuxeo-java-client/issues/21
Re: [ CONNECTORS-1290 ] Nuxeo connector for Apache ManifoldCF
Posted by David Arroyo <ar...@gmail.com>.
Hi Many thanks Karl. I have done a PR to nuxeo java client to fix the issue
and I hope it will be approved soon. When it be approved, I will finish the
connector and I will talk with Rafa to integrate it properly. Regards,
David.
On 22 February 2017 at 09:55, Rafa Haro <rh...@apache.org> wrote:
> Hi Karl,
>
> Thanks for checking this and making the proper changes. I will take care of
> updating the Nuxeo connector branch to current trunk version so David can
> safely now integrate the connector there.
>
> Thanks to both!
>
> On Wed, Feb 22, 2017 at 2:22 AM Karl Wright <da...@gmail.com> wrote:
>
> > Ok, committed. For the record:
> >
> > - Wiremock updated to 2.5.1 (which has jackson 2.6.1 dependencies)
> > - Amazon Cloud Search Connector jackson dependencies updated to 2.6.1.
> > - Tika Transformer jackson dependencies removed.
> > - No changes to Amazon S3 connector for now.
> >
> > Karl
> >
> >
> > On Tue, Feb 21, 2017 at 7:31 PM, Karl Wright <da...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I did an analysis, with the help of mvn dependency:tree, and determined
> > > the following usages of these:
> > >
> > > (1) wiremock, which is used for unit tests only; our version
> > > is com.github.tomakehurst:wiremock:jar:1.34 - we should see if we can
> > > upgrade this and still have things work. I'll check it out.
> > > (2) A direct dependency from the Amazon Cloud Search connector on
> version
> > > 2.3.x. I'm not sure why there's a direct dependency. We should try to
> > > build without the dependency there at all. I'll look into it.
> > > (3) A direct dependency in the Tika Transformer on 2.3.x. I'll look
> into
> > > that too.
> > > (4) A child dependency of com.amazonaws:aws-java-sdk-core:jar:1.9.0,
> on
> > > 2.1.2, used for the Amazon S3 connector. Maybe that can be upgraded?
> > >
> > > That's it. I'll do some research and see if I can bump the versions of
> > > these libraries up to something more modern.
> > >
> > > Karl
> > >
> > >
> > > On Tue, Feb 21, 2017 at 6:51 PM, Karl Wright <da...@gmail.com>
> wrote:
> > >
> > >> Hi David,
> > >>
> > >> If the jackson libraries make an attempt to preserve backwards
> > >> compatibility, it should be fine to update to 2.6.0. I don't recall
> the
> > >> connectors that make use of those (maybe they were a dependency of
> > Tika?)
> > >> but in general, well-behaved libraries do not change their APIs
> > >> carelessly. Of course, you need to confirm this picture as best you
> > can:
> > >> change the dependencies, download them (using ant make-core-deps)
> build,
> > >> and run tests. The tests, though, do not exercise all connectors, so
> it
> > >> would help to determine where the dependency came from and if
> reasonable
> > >> make sure those connectors still work after the update.
> > >>
> > >> Thanks,
> > >> Karl
> > >>
> > >>
> > >> On Tue, Feb 21, 2017 at 6:27 PM, David Arroyo <
> > >> arroyoescobardavid@gmail.com> wrote:
> > >>
> > >>> Hello everyone
> > >>>
> > >>>
> > >>> I'm updating the Nuxeo connector with the nuxeo java client. I have
> > found
> > >>> an issue with the libraries jackson-annotations and jackson-databind
> > due
> > >>> to
> > >>> monifoldcf uses versions 2.1.1 and 2.1.3 respectively, and the nuxeo
> > java
> > >>> client needs at least 2.6.0 version for both.
> > >>>
> > >>>
> > >>> If i compile and build the last manifold with the nexus connector, it
> > >>> works
> > >>> fine, but I have not tested it with other connectors. Is there any
> > >>> problem
> > >>> for update the libraries to 2.6.0 version?
> > >>>
> > >>>
> > >>> Moreover, I am adding new functionality to connector for filter the
> > type
> > >>> of
> > >>> files we want to index.
> > >>>
> > >>>
> > >>> Aside of that, nuxeo java client seems to have a bug for recovery the
> > >>> attachment content, so it can not be recovered. Therefore, I think
> > there
> > >>> are 3 possible options:
> > >>>
> > >>> 1. Wait the issue be fixed [1]
> > >>>
> > >>> 2. Generate a patch for the nuxeo client (Not sure how to share this
> > >>> patched lib)
> > >>>
> > >>> 3. Not included the content of the attachments
> > >>>
> > >>>
> > >>> Regards.
> > >>>
> > >>>
> > >>> [1] https://github.com/nuxeo/nuxeo-java-client/issues/21
> > >>>
> > >>
> > >>
> > >
> >
>
--
David Arroyo Escobar
Re: [ CONNECTORS-1290 ] Nuxeo connector for Apache ManifoldCF
Posted by Rafa Haro <rh...@apache.org>.
Hi Karl,
Thanks for checking this and making the proper changes. I will take care of
updating the Nuxeo connector branch to current trunk version so David can
safely now integrate the connector there.
Thanks to both!
On Wed, Feb 22, 2017 at 2:22 AM Karl Wright <da...@gmail.com> wrote:
> Ok, committed. For the record:
>
> - Wiremock updated to 2.5.1 (which has jackson 2.6.1 dependencies)
> - Amazon Cloud Search Connector jackson dependencies updated to 2.6.1.
> - Tika Transformer jackson dependencies removed.
> - No changes to Amazon S3 connector for now.
>
> Karl
>
>
> On Tue, Feb 21, 2017 at 7:31 PM, Karl Wright <da...@gmail.com> wrote:
>
> > Hi,
> >
> > I did an analysis, with the help of mvn dependency:tree, and determined
> > the following usages of these:
> >
> > (1) wiremock, which is used for unit tests only; our version
> > is com.github.tomakehurst:wiremock:jar:1.34 - we should see if we can
> > upgrade this and still have things work. I'll check it out.
> > (2) A direct dependency from the Amazon Cloud Search connector on version
> > 2.3.x. I'm not sure why there's a direct dependency. We should try to
> > build without the dependency there at all. I'll look into it.
> > (3) A direct dependency in the Tika Transformer on 2.3.x. I'll look into
> > that too.
> > (4) A child dependency of com.amazonaws:aws-java-sdk-core:jar:1.9.0, on
> > 2.1.2, used for the Amazon S3 connector. Maybe that can be upgraded?
> >
> > That's it. I'll do some research and see if I can bump the versions of
> > these libraries up to something more modern.
> >
> > Karl
> >
> >
> > On Tue, Feb 21, 2017 at 6:51 PM, Karl Wright <da...@gmail.com> wrote:
> >
> >> Hi David,
> >>
> >> If the jackson libraries make an attempt to preserve backwards
> >> compatibility, it should be fine to update to 2.6.0. I don't recall the
> >> connectors that make use of those (maybe they were a dependency of
> Tika?)
> >> but in general, well-behaved libraries do not change their APIs
> >> carelessly. Of course, you need to confirm this picture as best you
> can:
> >> change the dependencies, download them (using ant make-core-deps) build,
> >> and run tests. The tests, though, do not exercise all connectors, so it
> >> would help to determine where the dependency came from and if reasonable
> >> make sure those connectors still work after the update.
> >>
> >> Thanks,
> >> Karl
> >>
> >>
> >> On Tue, Feb 21, 2017 at 6:27 PM, David Arroyo <
> >> arroyoescobardavid@gmail.com> wrote:
> >>
> >>> Hello everyone
> >>>
> >>>
> >>> I'm updating the Nuxeo connector with the nuxeo java client. I have
> found
> >>> an issue with the libraries jackson-annotations and jackson-databind
> due
> >>> to
> >>> monifoldcf uses versions 2.1.1 and 2.1.3 respectively, and the nuxeo
> java
> >>> client needs at least 2.6.0 version for both.
> >>>
> >>>
> >>> If i compile and build the last manifold with the nexus connector, it
> >>> works
> >>> fine, but I have not tested it with other connectors. Is there any
> >>> problem
> >>> for update the libraries to 2.6.0 version?
> >>>
> >>>
> >>> Moreover, I am adding new functionality to connector for filter the
> type
> >>> of
> >>> files we want to index.
> >>>
> >>>
> >>> Aside of that, nuxeo java client seems to have a bug for recovery the
> >>> attachment content, so it can not be recovered. Therefore, I think
> there
> >>> are 3 possible options:
> >>>
> >>> 1. Wait the issue be fixed [1]
> >>>
> >>> 2. Generate a patch for the nuxeo client (Not sure how to share this
> >>> patched lib)
> >>>
> >>> 3. Not included the content of the attachments
> >>>
> >>>
> >>> Regards.
> >>>
> >>>
> >>> [1] https://github.com/nuxeo/nuxeo-java-client/issues/21
> >>>
> >>
> >>
> >
>
Re: [ CONNECTORS-1290 ] Nuxeo connector for Apache ManifoldCF
Posted by Karl Wright <da...@gmail.com>.
Ok, committed. For the record:
- Wiremock updated to 2.5.1 (which has jackson 2.6.1 dependencies)
- Amazon Cloud Search Connector jackson dependencies updated to 2.6.1.
- Tika Transformer jackson dependencies removed.
- No changes to Amazon S3 connector for now.
Karl
On Tue, Feb 21, 2017 at 7:31 PM, Karl Wright <da...@gmail.com> wrote:
> Hi,
>
> I did an analysis, with the help of mvn dependency:tree, and determined
> the following usages of these:
>
> (1) wiremock, which is used for unit tests only; our version
> is com.github.tomakehurst:wiremock:jar:1.34 - we should see if we can
> upgrade this and still have things work. I'll check it out.
> (2) A direct dependency from the Amazon Cloud Search connector on version
> 2.3.x. I'm not sure why there's a direct dependency. We should try to
> build without the dependency there at all. I'll look into it.
> (3) A direct dependency in the Tika Transformer on 2.3.x. I'll look into
> that too.
> (4) A child dependency of com.amazonaws:aws-java-sdk-core:jar:1.9.0, on
> 2.1.2, used for the Amazon S3 connector. Maybe that can be upgraded?
>
> That's it. I'll do some research and see if I can bump the versions of
> these libraries up to something more modern.
>
> Karl
>
>
> On Tue, Feb 21, 2017 at 6:51 PM, Karl Wright <da...@gmail.com> wrote:
>
>> Hi David,
>>
>> If the jackson libraries make an attempt to preserve backwards
>> compatibility, it should be fine to update to 2.6.0. I don't recall the
>> connectors that make use of those (maybe they were a dependency of Tika?)
>> but in general, well-behaved libraries do not change their APIs
>> carelessly. Of course, you need to confirm this picture as best you can:
>> change the dependencies, download them (using ant make-core-deps) build,
>> and run tests. The tests, though, do not exercise all connectors, so it
>> would help to determine where the dependency came from and if reasonable
>> make sure those connectors still work after the update.
>>
>> Thanks,
>> Karl
>>
>>
>> On Tue, Feb 21, 2017 at 6:27 PM, David Arroyo <
>> arroyoescobardavid@gmail.com> wrote:
>>
>>> Hello everyone
>>>
>>>
>>> I'm updating the Nuxeo connector with the nuxeo java client. I have found
>>> an issue with the libraries jackson-annotations and jackson-databind due
>>> to
>>> monifoldcf uses versions 2.1.1 and 2.1.3 respectively, and the nuxeo java
>>> client needs at least 2.6.0 version for both.
>>>
>>>
>>> If i compile and build the last manifold with the nexus connector, it
>>> works
>>> fine, but I have not tested it with other connectors. Is there any
>>> problem
>>> for update the libraries to 2.6.0 version?
>>>
>>>
>>> Moreover, I am adding new functionality to connector for filter the type
>>> of
>>> files we want to index.
>>>
>>>
>>> Aside of that, nuxeo java client seems to have a bug for recovery the
>>> attachment content, so it can not be recovered. Therefore, I think there
>>> are 3 possible options:
>>>
>>> 1. Wait the issue be fixed [1]
>>>
>>> 2. Generate a patch for the nuxeo client (Not sure how to share this
>>> patched lib)
>>>
>>> 3. Not included the content of the attachments
>>>
>>>
>>> Regards.
>>>
>>>
>>> [1] https://github.com/nuxeo/nuxeo-java-client/issues/21
>>>
>>
>>
>
Re: [ CONNECTORS-1290 ] Nuxeo connector for Apache ManifoldCF
Posted by Karl Wright <da...@gmail.com>.
It looks like I can upgrade them safely. I'm also going to wiremock 2.5.1
while I'm at it. The only one I'm not upgrading is the Amazon s3
Connector, because the newer SDK has a number of new dependencies and I'd
rather not risk it without a test.
I'll commit shortly.
Thanks,
Karl
On Tue, Feb 21, 2017 at 7:31 PM, Karl Wright <da...@gmail.com> wrote:
> Hi,
>
> I did an analysis, with the help of mvn dependency:tree, and determined
> the following usages of these:
>
> (1) wiremock, which is used for unit tests only; our version
> is com.github.tomakehurst:wiremock:jar:1.34 - we should see if we can
> upgrade this and still have things work. I'll check it out.
> (2) A direct dependency from the Amazon Cloud Search connector on version
> 2.3.x. I'm not sure why there's a direct dependency. We should try to
> build without the dependency there at all. I'll look into it.
> (3) A direct dependency in the Tika Transformer on 2.3.x. I'll look into
> that too.
> (4) A child dependency of com.amazonaws:aws-java-sdk-core:jar:1.9.0, on
> 2.1.2, used for the Amazon S3 connector. Maybe that can be upgraded?
>
> That's it. I'll do some research and see if I can bump the versions of
> these libraries up to something more modern.
>
> Karl
>
>
> On Tue, Feb 21, 2017 at 6:51 PM, Karl Wright <da...@gmail.com> wrote:
>
>> Hi David,
>>
>> If the jackson libraries make an attempt to preserve backwards
>> compatibility, it should be fine to update to 2.6.0. I don't recall the
>> connectors that make use of those (maybe they were a dependency of Tika?)
>> but in general, well-behaved libraries do not change their APIs
>> carelessly. Of course, you need to confirm this picture as best you can:
>> change the dependencies, download them (using ant make-core-deps) build,
>> and run tests. The tests, though, do not exercise all connectors, so it
>> would help to determine where the dependency came from and if reasonable
>> make sure those connectors still work after the update.
>>
>> Thanks,
>> Karl
>>
>>
>> On Tue, Feb 21, 2017 at 6:27 PM, David Arroyo <
>> arroyoescobardavid@gmail.com> wrote:
>>
>>> Hello everyone
>>>
>>>
>>> I'm updating the Nuxeo connector with the nuxeo java client. I have found
>>> an issue with the libraries jackson-annotations and jackson-databind due
>>> to
>>> monifoldcf uses versions 2.1.1 and 2.1.3 respectively, and the nuxeo java
>>> client needs at least 2.6.0 version for both.
>>>
>>>
>>> If i compile and build the last manifold with the nexus connector, it
>>> works
>>> fine, but I have not tested it with other connectors. Is there any
>>> problem
>>> for update the libraries to 2.6.0 version?
>>>
>>>
>>> Moreover, I am adding new functionality to connector for filter the type
>>> of
>>> files we want to index.
>>>
>>>
>>> Aside of that, nuxeo java client seems to have a bug for recovery the
>>> attachment content, so it can not be recovered. Therefore, I think there
>>> are 3 possible options:
>>>
>>> 1. Wait the issue be fixed [1]
>>>
>>> 2. Generate a patch for the nuxeo client (Not sure how to share this
>>> patched lib)
>>>
>>> 3. Not included the content of the attachments
>>>
>>>
>>> Regards.
>>>
>>>
>>> [1] https://github.com/nuxeo/nuxeo-java-client/issues/21
>>>
>>
>>
>
Re: [ CONNECTORS-1290 ] Nuxeo connector for Apache ManifoldCF
Posted by Karl Wright <da...@gmail.com>.
Hi,
I did an analysis, with the help of mvn dependency:tree, and determined the
following usages of these:
(1) wiremock, which is used for unit tests only; our version
is com.github.tomakehurst:wiremock:jar:1.34 - we should see if we can
upgrade this and still have things work. I'll check it out.
(2) A direct dependency from the Amazon Cloud Search connector on version
2.3.x. I'm not sure why there's a direct dependency. We should try to
build without the dependency there at all. I'll look into it.
(3) A direct dependency in the Tika Transformer on 2.3.x. I'll look into
that too.
(4) A child dependency of com.amazonaws:aws-java-sdk-core:jar:1.9.0, on
2.1.2, used for the Amazon S3 connector. Maybe that can be upgraded?
That's it. I'll do some research and see if I can bump the versions of
these libraries up to something more modern.
Karl
On Tue, Feb 21, 2017 at 6:51 PM, Karl Wright <da...@gmail.com> wrote:
> Hi David,
>
> If the jackson libraries make an attempt to preserve backwards
> compatibility, it should be fine to update to 2.6.0. I don't recall the
> connectors that make use of those (maybe they were a dependency of Tika?)
> but in general, well-behaved libraries do not change their APIs
> carelessly. Of course, you need to confirm this picture as best you can:
> change the dependencies, download them (using ant make-core-deps) build,
> and run tests. The tests, though, do not exercise all connectors, so it
> would help to determine where the dependency came from and if reasonable
> make sure those connectors still work after the update.
>
> Thanks,
> Karl
>
>
> On Tue, Feb 21, 2017 at 6:27 PM, David Arroyo <
> arroyoescobardavid@gmail.com> wrote:
>
>> Hello everyone
>>
>>
>> I'm updating the Nuxeo connector with the nuxeo java client. I have found
>> an issue with the libraries jackson-annotations and jackson-databind due
>> to
>> monifoldcf uses versions 2.1.1 and 2.1.3 respectively, and the nuxeo java
>> client needs at least 2.6.0 version for both.
>>
>>
>> If i compile and build the last manifold with the nexus connector, it
>> works
>> fine, but I have not tested it with other connectors. Is there any problem
>> for update the libraries to 2.6.0 version?
>>
>>
>> Moreover, I am adding new functionality to connector for filter the type
>> of
>> files we want to index.
>>
>>
>> Aside of that, nuxeo java client seems to have a bug for recovery the
>> attachment content, so it can not be recovered. Therefore, I think there
>> are 3 possible options:
>>
>> 1. Wait the issue be fixed [1]
>>
>> 2. Generate a patch for the nuxeo client (Not sure how to share this
>> patched lib)
>>
>> 3. Not included the content of the attachments
>>
>>
>> Regards.
>>
>>
>> [1] https://github.com/nuxeo/nuxeo-java-client/issues/21
>>
>
>
Re: [ CONNECTORS-1290 ] Nuxeo connector for Apache ManifoldCF
Posted by Karl Wright <da...@gmail.com>.
Hi David,
If the jackson libraries make an attempt to preserve backwards
compatibility, it should be fine to update to 2.6.0. I don't recall the
connectors that make use of those (maybe they were a dependency of Tika?)
but in general, well-behaved libraries do not change their APIs
carelessly. Of course, you need to confirm this picture as best you can:
change the dependencies, download them (using ant make-core-deps) build,
and run tests. The tests, though, do not exercise all connectors, so it
would help to determine where the dependency came from and if reasonable
make sure those connectors still work after the update.
Thanks,
Karl
On Tue, Feb 21, 2017 at 6:27 PM, David Arroyo <ar...@gmail.com>
wrote:
> Hello everyone
>
>
> I'm updating the Nuxeo connector with the nuxeo java client. I have found
> an issue with the libraries jackson-annotations and jackson-databind due to
> monifoldcf uses versions 2.1.1 and 2.1.3 respectively, and the nuxeo java
> client needs at least 2.6.0 version for both.
>
>
> If i compile and build the last manifold with the nexus connector, it works
> fine, but I have not tested it with other connectors. Is there any problem
> for update the libraries to 2.6.0 version?
>
>
> Moreover, I am adding new functionality to connector for filter the type of
> files we want to index.
>
>
> Aside of that, nuxeo java client seems to have a bug for recovery the
> attachment content, so it can not be recovered. Therefore, I think there
> are 3 possible options:
>
> 1. Wait the issue be fixed [1]
>
> 2. Generate a patch for the nuxeo client (Not sure how to share this
> patched lib)
>
> 3. Not included the content of the attachments
>
>
> Regards.
>
>
> [1] https://github.com/nuxeo/nuxeo-java-client/issues/21
>