You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by Piergiorgio Lucidi <pi...@apache.org> on 2014/05/26 12:00:38 UTC
MongoDB Repository and Output Connectors
Hi guys,
I think that it could be very useful to add in ManifoldCF both the
connectors related to MongoDB.
During these days I have taken a look at MongoDB and I think that I can
start to implement these connectors.
The repository connector could make sense because MongoDB is a document
repository so some people could need to create smart indexes in a search
server, I'm thinking about huge repositories.
The MongoDB search engine could be very slow in some scenarios compared to
the modern search engines.
The output connector could make sense because you probably want to convert
your wide data to a flat view in MongoDB to execute queries in a flat way.
Here you probably don't need performance but some good queries for analysis
to create your monthly or weekly reports with MongoDB.
I know that MongoDB is not a search server but it is used a lot for
creating dashboards and reports and the usage is similar to a search engine.
What do you think about this?
Please let me know.
Thank you all.
Cheers,
Piergiorgio
--
Piergiorgio Lucidi
Open Source ECM Specialist
http://www.open4dev.com
Re: MongoDB Repository and Output Connectors
Posted by Muhammed Olgun <mh...@gmail.com>.
Hi,
I worked a MongoDB related work too. It’s GridFS specific and I wrote a repository connector. I would like to share my code. May be we can combine it with MongoDB connector. They can be separate too.
https://github.com/molgun/MCF-GridFS-Connector
On 26 May 2014, at 15:42, Piergiorgio Lucidi <pi...@apache.org> wrote:
> Hi Karl,
>
>
> 2014-05-26 13:17 GMT+02:00 Wright, Karl <ka...@here.com>:
>
>> Hi Piergiorgio,
>>
>> When you say 'huge', how many documents do you mean? Do you know of any
>> specific installations, and if so, how many documents do they work with?
>>
>
>> I think it is fine to develop connectors for mongodb, but if the number of
>> documents is truly huge, there will need to be a way of dividing up the
>> task.
>>
>
> I imagine the typical usage of MongoDB, for example to keep User Generated
> Contents (UGC), this means that the information is just a little section of
> data for implementing social features dedicated to a website or a portal.
>
> I'm sorry I don't have real-world numbers now but we could estimate
> thousands of contents. Actually I'm working on a project that is starting
> to use MongoDB and the use case is UGC.
>
> Please consider that MongoDB divide its contents into databases, and for
> each database you have different collections of contents.
> This means that the connector should ask for a database and a specific
> collection to work, this because MongoDB doesn't support the join of data
> from different collections, it is not a relational database.
>
> So I think that it should be very easy to implement in ManifoldCF.
>
> Piergiorgio
>
>
>>
>> Karl
>>
>> Sent from my Windows Phone
>> ------------------------------
>> From: Piergiorgio Lucidi
>> Sent: 5/26/2014 6:01 AM
>> To: dev@manifoldcf.apache.org
>> Subject: MongoDB Repository and Output Connectors
>>
>> Hi guys,
>>
>> I think that it could be very useful to add in ManifoldCF both the
>> connectors related to MongoDB.
>> During these days I have taken a look at MongoDB and I think that I can
>> start to implement these connectors.
>>
>> The repository connector could make sense because MongoDB is a document
>> repository so some people could need to create smart indexes in a search
>> server, I'm thinking about huge repositories.
>> The MongoDB search engine could be very slow in some scenarios compared to
>> the modern search engines.
>>
>> The output connector could make sense because you probably want to convert
>> your wide data to a flat view in MongoDB to execute queries in a flat way.
>> Here you probably don't need performance but some good queries for analysis
>> to create your monthly or weekly reports with MongoDB.
>>
>> I know that MongoDB is not a search server but it is used a lot for
>> creating dashboards and reports and the usage is similar to a search
>> engine.
>>
>> What do you think about this?
>>
>> Please let me know.
>> Thank you all.
>>
>> Cheers,
>> Piergiorgio
>>
>> --
>> Piergiorgio Lucidi
>> Open Source ECM Specialist
>> http://www.open4dev.com
>>
>> --
>> <http://www.open4dev.com>
>> Piergiorgio Lucidi <http://www.open4dev.com>
>> Open Source ECM Specialist
>> <http://www.open4dev.com>http://www.open4dev.com
Re: MongoDB Repository and Output Connectors
Posted by Piergiorgio Lucidi <pi...@apache.org>.
Hi Karl,
2014-05-26 13:17 GMT+02:00 Wright, Karl <ka...@here.com>:
> Hi Piergiorgio,
>
> When you say 'huge', how many documents do you mean? Do you know of any
> specific installations, and if so, how many documents do they work with?
>
> I think it is fine to develop connectors for mongodb, but if the number of
> documents is truly huge, there will need to be a way of dividing up the
> task.
>
I imagine the typical usage of MongoDB, for example to keep User Generated
Contents (UGC), this means that the information is just a little section of
data for implementing social features dedicated to a website or a portal.
I'm sorry I don't have real-world numbers now but we could estimate
thousands of contents. Actually I'm working on a project that is starting
to use MongoDB and the use case is UGC.
Please consider that MongoDB divide its contents into databases, and for
each database you have different collections of contents.
This means that the connector should ask for a database and a specific
collection to work, this because MongoDB doesn't support the join of data
from different collections, it is not a relational database.
So I think that it should be very easy to implement in ManifoldCF.
Piergiorgio
>
> Karl
>
> Sent from my Windows Phone
> ------------------------------
> From: Piergiorgio Lucidi
> Sent: 5/26/2014 6:01 AM
> To: dev@manifoldcf.apache.org
> Subject: MongoDB Repository and Output Connectors
>
> Hi guys,
>
> I think that it could be very useful to add in ManifoldCF both the
> connectors related to MongoDB.
> During these days I have taken a look at MongoDB and I think that I can
> start to implement these connectors.
>
> The repository connector could make sense because MongoDB is a document
> repository so some people could need to create smart indexes in a search
> server, I'm thinking about huge repositories.
> The MongoDB search engine could be very slow in some scenarios compared to
> the modern search engines.
>
> The output connector could make sense because you probably want to convert
> your wide data to a flat view in MongoDB to execute queries in a flat way.
> Here you probably don't need performance but some good queries for analysis
> to create your monthly or weekly reports with MongoDB.
>
> I know that MongoDB is not a search server but it is used a lot for
> creating dashboards and reports and the usage is similar to a search
> engine.
>
> What do you think about this?
>
> Please let me know.
> Thank you all.
>
> Cheers,
> Piergiorgio
>
> --
> Piergiorgio Lucidi
> Open Source ECM Specialist
> http://www.open4dev.com
>
> --
> <http://www.open4dev.com>
> Piergiorgio Lucidi <http://www.open4dev.com>
> Open Source ECM Specialist
> <http://www.open4dev.com>http://www.open4dev.com
>
RE: MongoDB Repository and Output Connectors
Posted by "Wright, Karl" <ka...@here.com>.
Hi Piergiorgio,
When you say 'huge', how many documents do you mean? Do you know of any specific installations, and if so, how many documents do they work with?
I think it is fine to develop connectors for mongodb, but if the number of documents is truly huge, there will need to be a way of dividing up the task.
Karl
Sent from my Windows Phone
________________________________
From: Piergiorgio Lucidi
Sent: 5/26/2014 6:01 AM
To: dev@manifoldcf.apache.org
Subject: MongoDB Repository and Output Connectors
Hi guys,
I think that it could be very useful to add in ManifoldCF both the
connectors related to MongoDB.
During these days I have taken a look at MongoDB and I think that I can
start to implement these connectors.
The repository connector could make sense because MongoDB is a document
repository so some people could need to create smart indexes in a search
server, I'm thinking about huge repositories.
The MongoDB search engine could be very slow in some scenarios compared to
the modern search engines.
The output connector could make sense because you probably want to convert
your wide data to a flat view in MongoDB to execute queries in a flat way.
Here you probably don't need performance but some good queries for analysis
to create your monthly or weekly reports with MongoDB.
I know that MongoDB is not a search server but it is used a lot for
creating dashboards and reports and the usage is similar to a search engine.
What do you think about this?
Please let me know.
Thank you all.
Cheers,
Piergiorgio
--
Piergiorgio Lucidi
Open Source ECM Specialist
http://www.open4dev.com