You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@manifoldcf.apache.org by Ameya Aware <am...@gmail.com> on 2014/07/24 20:11:10 UTC
Performance on shared drive documents
Hi
When i am crawling full of my C drive the job is running pretty much faster.
But at the same time when i m crawling documents kept on shared drive, it
is taking too much longer time than to crawl C drive.
are documents at shared drive supposed to take to much time??
Thanks,
Ameya
Re: Performance on shared drive documents
Posted by Ameya Aware <am...@gmail.com>.
Sure. Thanks a lot Karl
On Thu, Jul 24, 2014 at 2:53 PM, Karl Wright <da...@gmail.com> wrote:
> Hi Ameya,
>
> ManifoldCF can only crawl as fast as the underlying repository will
> support. The performance in the case of a network drive is obviously going
> to be slower than that of a local drive.
>
> Karl
>
>
>
> On Thu, Jul 24, 2014 at 2:49 PM, Ameya Aware <am...@gmail.com>
> wrote:
>
>> Hi Karl,
>>
>> Currently document security is the not the main aim.
>>
>> I just want all the files to be crawled as quickly as possible.
>>
>> So do you want to say that my shared drive would never get crawled as
>> fast as my C drive?
>>
>>
>> On Thu, Jul 24, 2014 at 2:44 PM, Karl Wright <da...@gmail.com> wrote:
>>
>>> Hi Ameya,
>>>
>>> The filesystem connector does not support document security. Are you
>>> sure this is what you want to do?
>>>
>>> Either way, the same logic applies, since it is the same underlying
>>> technology.
>>>
>>> Karl
>>>
>>>
>>>
>>> On Thu, Jul 24, 2014 at 2:40 PM, Ameya Aware <am...@gmail.com>
>>> wrote:
>>>
>>>> i apologize for my mistake.
>>>>
>>>> I wanted to say that i am not using windows share connector. I am
>>>> using filesystem connector only where in i provide the location of shared
>>>> drive in repository paths.
>>>>
>>>>
>>>> On Thu, Jul 24, 2014 at 2:36 PM, Karl Wright <da...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Ameya,
>>>>>
>>>>> Yes, I figured that you would be using the share connector to crawl
>>>>> shared drives. That's what I am talking about, though. The share
>>>>> connector is only as fast as the server you are connected to, and there may
>>>>> be indirections also if the path contains DFS links. Regardless, though,
>>>>> Windows file servers are easily overloaded, in my experience, so try not to
>>>>> overload them unnecessarily. When a file server gets overloaded, it often
>>>>> fails with a timeout (which you can see in the simple history report, or in
>>>>> the manifoldcf log). That's a sign that you need to throttle more, and the
>>>>> best way to do that is to limit further the number of simultaneous
>>>>> connections.
>>>>>
>>>>> Thanks,
>>>>> Karl
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jul 24, 2014 at 2:32 PM, Ameya Aware <am...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Actually i am using windows share connector. I am using filesystem
>>>>>> connector only where in i provide the location of shared drive in
>>>>>> repository paths.
>>>>>>
>>>>>>
>>>>>> On Thu, Jul 24, 2014 at 2:13 PM, Karl Wright <da...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Ameya,
>>>>>>>
>>>>>>> Windows shared drive access performance is not great, I'm afraid.
>>>>>>> However, it is possible that you are overwhelming the server you are
>>>>>>> talking to. Try reducing the maximum number of connections for the shared
>>>>>>> drive repository connection to between 2 and 5; that sometimes helps.
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jul 24, 2014 at 2:11 PM, Ameya Aware <am...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> When i am crawling full of my C drive the job is running pretty
>>>>>>>> much faster.
>>>>>>>>
>>>>>>>> But at the same time when i m crawling documents kept on shared
>>>>>>>> drive, it is taking too much longer time than to crawl C drive.
>>>>>>>>
>>>>>>>> are documents at shared drive supposed to take to much time??
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ameya
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Performance on shared drive documents
Posted by Karl Wright <da...@gmail.com>.
Hi Ameya,
If reducing the number of simultaneous connections didn't help throughput,
then frankly the only way to make things faster is to improve the
performance of the CIFS drives you are crawling. You could try to purchase
dedicated NetApp devices for those, if that hasn't already been done.
Beyond that I have no solutions for you. See ManifoldCF In Action Chapter
1.
Karl
On Wed, Jul 30, 2014 at 11:18 AM, Ameya Aware <am...@gmail.com> wrote:
> So how can improve the performance of crawling for shared drives?
>
>
> Thanks,
> Ameya
>
>
> On Thu, Jul 24, 2014 at 2:53 PM, Karl Wright <da...@gmail.com> wrote:
>
>> Hi Ameya,
>>
>> ManifoldCF can only crawl as fast as the underlying repository will
>> support. The performance in the case of a network drive is obviously going
>> to be slower than that of a local drive.
>>
>> Karl
>>
>>
>>
>> On Thu, Jul 24, 2014 at 2:49 PM, Ameya Aware <am...@gmail.com>
>> wrote:
>>
>>> Hi Karl,
>>>
>>> Currently document security is the not the main aim.
>>>
>>> I just want all the files to be crawled as quickly as possible.
>>>
>>> So do you want to say that my shared drive would never get crawled as
>>> fast as my C drive?
>>>
>>>
>>> On Thu, Jul 24, 2014 at 2:44 PM, Karl Wright <da...@gmail.com> wrote:
>>>
>>>> Hi Ameya,
>>>>
>>>> The filesystem connector does not support document security. Are you
>>>> sure this is what you want to do?
>>>>
>>>> Either way, the same logic applies, since it is the same underlying
>>>> technology.
>>>>
>>>> Karl
>>>>
>>>>
>>>>
>>>> On Thu, Jul 24, 2014 at 2:40 PM, Ameya Aware <am...@gmail.com>
>>>> wrote:
>>>>
>>>>> i apologize for my mistake.
>>>>>
>>>>> I wanted to say that i am not using windows share connector. I am
>>>>> using filesystem connector only where in i provide the location of shared
>>>>> drive in repository paths.
>>>>>
>>>>>
>>>>> On Thu, Jul 24, 2014 at 2:36 PM, Karl Wright <da...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ameya,
>>>>>>
>>>>>> Yes, I figured that you would be using the share connector to crawl
>>>>>> shared drives. That's what I am talking about, though. The share
>>>>>> connector is only as fast as the server you are connected to, and there may
>>>>>> be indirections also if the path contains DFS links. Regardless, though,
>>>>>> Windows file servers are easily overloaded, in my experience, so try not to
>>>>>> overload them unnecessarily. When a file server gets overloaded, it often
>>>>>> fails with a timeout (which you can see in the simple history report, or in
>>>>>> the manifoldcf log). That's a sign that you need to throttle more, and the
>>>>>> best way to do that is to limit further the number of simultaneous
>>>>>> connections.
>>>>>>
>>>>>> Thanks,
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jul 24, 2014 at 2:32 PM, Ameya Aware <am...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Actually i am using windows share connector. I am using filesystem
>>>>>>> connector only where in i provide the location of shared drive in
>>>>>>> repository paths.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jul 24, 2014 at 2:13 PM, Karl Wright <da...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Ameya,
>>>>>>>>
>>>>>>>> Windows shared drive access performance is not great, I'm afraid.
>>>>>>>> However, it is possible that you are overwhelming the server you are
>>>>>>>> talking to. Try reducing the maximum number of connections for the shared
>>>>>>>> drive repository connection to between 2 and 5; that sometimes helps.
>>>>>>>>
>>>>>>>> Karl
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jul 24, 2014 at 2:11 PM, Ameya Aware <ameya.aware@gmail.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> When i am crawling full of my C drive the job is running pretty
>>>>>>>>> much faster.
>>>>>>>>>
>>>>>>>>> But at the same time when i m crawling documents kept on shared
>>>>>>>>> drive, it is taking too much longer time than to crawl C drive.
>>>>>>>>>
>>>>>>>>> are documents at shared drive supposed to take to much time??
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Ameya
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Performance on shared drive documents
Posted by Ameya Aware <am...@gmail.com>.
So how can improve the performance of crawling for shared drives?
Thanks,
Ameya
On Thu, Jul 24, 2014 at 2:53 PM, Karl Wright <da...@gmail.com> wrote:
> Hi Ameya,
>
> ManifoldCF can only crawl as fast as the underlying repository will
> support. The performance in the case of a network drive is obviously going
> to be slower than that of a local drive.
>
> Karl
>
>
>
> On Thu, Jul 24, 2014 at 2:49 PM, Ameya Aware <am...@gmail.com>
> wrote:
>
>> Hi Karl,
>>
>> Currently document security is the not the main aim.
>>
>> I just want all the files to be crawled as quickly as possible.
>>
>> So do you want to say that my shared drive would never get crawled as
>> fast as my C drive?
>>
>>
>> On Thu, Jul 24, 2014 at 2:44 PM, Karl Wright <da...@gmail.com> wrote:
>>
>>> Hi Ameya,
>>>
>>> The filesystem connector does not support document security. Are you
>>> sure this is what you want to do?
>>>
>>> Either way, the same logic applies, since it is the same underlying
>>> technology.
>>>
>>> Karl
>>>
>>>
>>>
>>> On Thu, Jul 24, 2014 at 2:40 PM, Ameya Aware <am...@gmail.com>
>>> wrote:
>>>
>>>> i apologize for my mistake.
>>>>
>>>> I wanted to say that i am not using windows share connector. I am
>>>> using filesystem connector only where in i provide the location of shared
>>>> drive in repository paths.
>>>>
>>>>
>>>> On Thu, Jul 24, 2014 at 2:36 PM, Karl Wright <da...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Ameya,
>>>>>
>>>>> Yes, I figured that you would be using the share connector to crawl
>>>>> shared drives. That's what I am talking about, though. The share
>>>>> connector is only as fast as the server you are connected to, and there may
>>>>> be indirections also if the path contains DFS links. Regardless, though,
>>>>> Windows file servers are easily overloaded, in my experience, so try not to
>>>>> overload them unnecessarily. When a file server gets overloaded, it often
>>>>> fails with a timeout (which you can see in the simple history report, or in
>>>>> the manifoldcf log). That's a sign that you need to throttle more, and the
>>>>> best way to do that is to limit further the number of simultaneous
>>>>> connections.
>>>>>
>>>>> Thanks,
>>>>> Karl
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jul 24, 2014 at 2:32 PM, Ameya Aware <am...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Actually i am using windows share connector. I am using filesystem
>>>>>> connector only where in i provide the location of shared drive in
>>>>>> repository paths.
>>>>>>
>>>>>>
>>>>>> On Thu, Jul 24, 2014 at 2:13 PM, Karl Wright <da...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Ameya,
>>>>>>>
>>>>>>> Windows shared drive access performance is not great, I'm afraid.
>>>>>>> However, it is possible that you are overwhelming the server you are
>>>>>>> talking to. Try reducing the maximum number of connections for the shared
>>>>>>> drive repository connection to between 2 and 5; that sometimes helps.
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jul 24, 2014 at 2:11 PM, Ameya Aware <am...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> When i am crawling full of my C drive the job is running pretty
>>>>>>>> much faster.
>>>>>>>>
>>>>>>>> But at the same time when i m crawling documents kept on shared
>>>>>>>> drive, it is taking too much longer time than to crawl C drive.
>>>>>>>>
>>>>>>>> are documents at shared drive supposed to take to much time??
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ameya
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Performance on shared drive documents
Posted by Karl Wright <da...@gmail.com>.
Hi Ameya,
ManifoldCF can only crawl as fast as the underlying repository will
support. The performance in the case of a network drive is obviously going
to be slower than that of a local drive.
Karl
On Thu, Jul 24, 2014 at 2:49 PM, Ameya Aware <am...@gmail.com> wrote:
> Hi Karl,
>
> Currently document security is the not the main aim.
>
> I just want all the files to be crawled as quickly as possible.
>
> So do you want to say that my shared drive would never get crawled as fast
> as my C drive?
>
>
> On Thu, Jul 24, 2014 at 2:44 PM, Karl Wright <da...@gmail.com> wrote:
>
>> Hi Ameya,
>>
>> The filesystem connector does not support document security. Are you
>> sure this is what you want to do?
>>
>> Either way, the same logic applies, since it is the same underlying
>> technology.
>>
>> Karl
>>
>>
>>
>> On Thu, Jul 24, 2014 at 2:40 PM, Ameya Aware <am...@gmail.com>
>> wrote:
>>
>>> i apologize for my mistake.
>>>
>>> I wanted to say that i am not using windows share connector. I am using
>>> filesystem connector only where in i provide the location of shared drive
>>> in repository paths.
>>>
>>>
>>> On Thu, Jul 24, 2014 at 2:36 PM, Karl Wright <da...@gmail.com> wrote:
>>>
>>>> Hi Ameya,
>>>>
>>>> Yes, I figured that you would be using the share connector to crawl
>>>> shared drives. That's what I am talking about, though. The share
>>>> connector is only as fast as the server you are connected to, and there may
>>>> be indirections also if the path contains DFS links. Regardless, though,
>>>> Windows file servers are easily overloaded, in my experience, so try not to
>>>> overload them unnecessarily. When a file server gets overloaded, it often
>>>> fails with a timeout (which you can see in the simple history report, or in
>>>> the manifoldcf log). That's a sign that you need to throttle more, and the
>>>> best way to do that is to limit further the number of simultaneous
>>>> connections.
>>>>
>>>> Thanks,
>>>> Karl
>>>>
>>>>
>>>>
>>>> On Thu, Jul 24, 2014 at 2:32 PM, Ameya Aware <am...@gmail.com>
>>>> wrote:
>>>>
>>>>> Actually i am using windows share connector. I am using filesystem
>>>>> connector only where in i provide the location of shared drive in
>>>>> repository paths.
>>>>>
>>>>>
>>>>> On Thu, Jul 24, 2014 at 2:13 PM, Karl Wright <da...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ameya,
>>>>>>
>>>>>> Windows shared drive access performance is not great, I'm afraid.
>>>>>> However, it is possible that you are overwhelming the server you are
>>>>>> talking to. Try reducing the maximum number of connections for the shared
>>>>>> drive repository connection to between 2 and 5; that sometimes helps.
>>>>>>
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jul 24, 2014 at 2:11 PM, Ameya Aware <am...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> When i am crawling full of my C drive the job is running pretty much
>>>>>>> faster.
>>>>>>>
>>>>>>> But at the same time when i m crawling documents kept on shared
>>>>>>> drive, it is taking too much longer time than to crawl C drive.
>>>>>>>
>>>>>>> are documents at shared drive supposed to take to much time??
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ameya
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Performance on shared drive documents
Posted by Ameya Aware <am...@gmail.com>.
Hi Karl,
Currently document security is the not the main aim.
I just want all the files to be crawled as quickly as possible.
So do you want to say that my shared drive would never get crawled as fast
as my C drive?
On Thu, Jul 24, 2014 at 2:44 PM, Karl Wright <da...@gmail.com> wrote:
> Hi Ameya,
>
> The filesystem connector does not support document security. Are you sure
> this is what you want to do?
>
> Either way, the same logic applies, since it is the same underlying
> technology.
>
> Karl
>
>
>
> On Thu, Jul 24, 2014 at 2:40 PM, Ameya Aware <am...@gmail.com>
> wrote:
>
>> i apologize for my mistake.
>>
>> I wanted to say that i am not using windows share connector. I am using
>> filesystem connector only where in i provide the location of shared drive
>> in repository paths.
>>
>>
>> On Thu, Jul 24, 2014 at 2:36 PM, Karl Wright <da...@gmail.com> wrote:
>>
>>> Hi Ameya,
>>>
>>> Yes, I figured that you would be using the share connector to crawl
>>> shared drives. That's what I am talking about, though. The share
>>> connector is only as fast as the server you are connected to, and there may
>>> be indirections also if the path contains DFS links. Regardless, though,
>>> Windows file servers are easily overloaded, in my experience, so try not to
>>> overload them unnecessarily. When a file server gets overloaded, it often
>>> fails with a timeout (which you can see in the simple history report, or in
>>> the manifoldcf log). That's a sign that you need to throttle more, and the
>>> best way to do that is to limit further the number of simultaneous
>>> connections.
>>>
>>> Thanks,
>>> Karl
>>>
>>>
>>>
>>> On Thu, Jul 24, 2014 at 2:32 PM, Ameya Aware <am...@gmail.com>
>>> wrote:
>>>
>>>> Actually i am using windows share connector. I am using filesystem
>>>> connector only where in i provide the location of shared drive in
>>>> repository paths.
>>>>
>>>>
>>>> On Thu, Jul 24, 2014 at 2:13 PM, Karl Wright <da...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Ameya,
>>>>>
>>>>> Windows shared drive access performance is not great, I'm afraid.
>>>>> However, it is possible that you are overwhelming the server you are
>>>>> talking to. Try reducing the maximum number of connections for the shared
>>>>> drive repository connection to between 2 and 5; that sometimes helps.
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jul 24, 2014 at 2:11 PM, Ameya Aware <am...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> When i am crawling full of my C drive the job is running pretty much
>>>>>> faster.
>>>>>>
>>>>>> But at the same time when i m crawling documents kept on shared
>>>>>> drive, it is taking too much longer time than to crawl C drive.
>>>>>>
>>>>>> are documents at shared drive supposed to take to much time??
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Ameya
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Performance on shared drive documents
Posted by Karl Wright <da...@gmail.com>.
Hi Ameya,
The filesystem connector does not support document security. Are you sure
this is what you want to do?
Either way, the same logic applies, since it is the same underlying
technology.
Karl
On Thu, Jul 24, 2014 at 2:40 PM, Ameya Aware <am...@gmail.com> wrote:
> i apologize for my mistake.
>
> I wanted to say that i am not using windows share connector. I am using
> filesystem connector only where in i provide the location of shared drive
> in repository paths.
>
>
> On Thu, Jul 24, 2014 at 2:36 PM, Karl Wright <da...@gmail.com> wrote:
>
>> Hi Ameya,
>>
>> Yes, I figured that you would be using the share connector to crawl
>> shared drives. That's what I am talking about, though. The share
>> connector is only as fast as the server you are connected to, and there may
>> be indirections also if the path contains DFS links. Regardless, though,
>> Windows file servers are easily overloaded, in my experience, so try not to
>> overload them unnecessarily. When a file server gets overloaded, it often
>> fails with a timeout (which you can see in the simple history report, or in
>> the manifoldcf log). That's a sign that you need to throttle more, and the
>> best way to do that is to limit further the number of simultaneous
>> connections.
>>
>> Thanks,
>> Karl
>>
>>
>>
>> On Thu, Jul 24, 2014 at 2:32 PM, Ameya Aware <am...@gmail.com>
>> wrote:
>>
>>> Actually i am using windows share connector. I am using filesystem
>>> connector only where in i provide the location of shared drive in
>>> repository paths.
>>>
>>>
>>> On Thu, Jul 24, 2014 at 2:13 PM, Karl Wright <da...@gmail.com> wrote:
>>>
>>>> Hi Ameya,
>>>>
>>>> Windows shared drive access performance is not great, I'm afraid.
>>>> However, it is possible that you are overwhelming the server you are
>>>> talking to. Try reducing the maximum number of connections for the shared
>>>> drive repository connection to between 2 and 5; that sometimes helps.
>>>>
>>>> Karl
>>>>
>>>>
>>>>
>>>> On Thu, Jul 24, 2014 at 2:11 PM, Ameya Aware <am...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> When i am crawling full of my C drive the job is running pretty much
>>>>> faster.
>>>>>
>>>>> But at the same time when i m crawling documents kept on shared drive,
>>>>> it is taking too much longer time than to crawl C drive.
>>>>>
>>>>> are documents at shared drive supposed to take to much time??
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Ameya
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Performance on shared drive documents
Posted by Ameya Aware <am...@gmail.com>.
i apologize for my mistake.
I wanted to say that i am not using windows share connector. I am using
filesystem connector only where in i provide the location of shared drive
in repository paths.
On Thu, Jul 24, 2014 at 2:36 PM, Karl Wright <da...@gmail.com> wrote:
> Hi Ameya,
>
> Yes, I figured that you would be using the share connector to crawl shared
> drives. That's what I am talking about, though. The share connector is
> only as fast as the server you are connected to, and there may be
> indirections also if the path contains DFS links. Regardless, though,
> Windows file servers are easily overloaded, in my experience, so try not to
> overload them unnecessarily. When a file server gets overloaded, it often
> fails with a timeout (which you can see in the simple history report, or in
> the manifoldcf log). That's a sign that you need to throttle more, and the
> best way to do that is to limit further the number of simultaneous
> connections.
>
> Thanks,
> Karl
>
>
>
> On Thu, Jul 24, 2014 at 2:32 PM, Ameya Aware <am...@gmail.com>
> wrote:
>
>> Actually i am using windows share connector. I am using filesystem
>> connector only where in i provide the location of shared drive in
>> repository paths.
>>
>>
>> On Thu, Jul 24, 2014 at 2:13 PM, Karl Wright <da...@gmail.com> wrote:
>>
>>> Hi Ameya,
>>>
>>> Windows shared drive access performance is not great, I'm afraid.
>>> However, it is possible that you are overwhelming the server you are
>>> talking to. Try reducing the maximum number of connections for the shared
>>> drive repository connection to between 2 and 5; that sometimes helps.
>>>
>>> Karl
>>>
>>>
>>>
>>> On Thu, Jul 24, 2014 at 2:11 PM, Ameya Aware <am...@gmail.com>
>>> wrote:
>>>
>>>> Hi
>>>>
>>>> When i am crawling full of my C drive the job is running pretty much
>>>> faster.
>>>>
>>>> But at the same time when i m crawling documents kept on shared drive,
>>>> it is taking too much longer time than to crawl C drive.
>>>>
>>>> are documents at shared drive supposed to take to much time??
>>>>
>>>>
>>>> Thanks,
>>>> Ameya
>>>>
>>>
>>>
>>
>
Re: Performance on shared drive documents
Posted by Karl Wright <da...@gmail.com>.
Hi Ameya,
Yes, I figured that you would be using the share connector to crawl shared
drives. That's what I am talking about, though. The share connector is
only as fast as the server you are connected to, and there may be
indirections also if the path contains DFS links. Regardless, though,
Windows file servers are easily overloaded, in my experience, so try not to
overload them unnecessarily. When a file server gets overloaded, it often
fails with a timeout (which you can see in the simple history report, or in
the manifoldcf log). That's a sign that you need to throttle more, and the
best way to do that is to limit further the number of simultaneous
connections.
Thanks,
Karl
On Thu, Jul 24, 2014 at 2:32 PM, Ameya Aware <am...@gmail.com> wrote:
> Actually i am using windows share connector. I am using filesystem
> connector only where in i provide the location of shared drive in
> repository paths.
>
>
> On Thu, Jul 24, 2014 at 2:13 PM, Karl Wright <da...@gmail.com> wrote:
>
>> Hi Ameya,
>>
>> Windows shared drive access performance is not great, I'm afraid.
>> However, it is possible that you are overwhelming the server you are
>> talking to. Try reducing the maximum number of connections for the shared
>> drive repository connection to between 2 and 5; that sometimes helps.
>>
>> Karl
>>
>>
>>
>> On Thu, Jul 24, 2014 at 2:11 PM, Ameya Aware <am...@gmail.com>
>> wrote:
>>
>>> Hi
>>>
>>> When i am crawling full of my C drive the job is running pretty much
>>> faster.
>>>
>>> But at the same time when i m crawling documents kept on shared drive,
>>> it is taking too much longer time than to crawl C drive.
>>>
>>> are documents at shared drive supposed to take to much time??
>>>
>>>
>>> Thanks,
>>> Ameya
>>>
>>
>>
>
Re: Performance on shared drive documents
Posted by Ameya Aware <am...@gmail.com>.
Actually i am using windows share connector. I am using filesystem
connector only where in i provide the location of shared drive in
repository paths.
On Thu, Jul 24, 2014 at 2:13 PM, Karl Wright <da...@gmail.com> wrote:
> Hi Ameya,
>
> Windows shared drive access performance is not great, I'm afraid.
> However, it is possible that you are overwhelming the server you are
> talking to. Try reducing the maximum number of connections for the shared
> drive repository connection to between 2 and 5; that sometimes helps.
>
> Karl
>
>
>
> On Thu, Jul 24, 2014 at 2:11 PM, Ameya Aware <am...@gmail.com>
> wrote:
>
>> Hi
>>
>> When i am crawling full of my C drive the job is running pretty much
>> faster.
>>
>> But at the same time when i m crawling documents kept on shared drive, it
>> is taking too much longer time than to crawl C drive.
>>
>> are documents at shared drive supposed to take to much time??
>>
>>
>> Thanks,
>> Ameya
>>
>
>
Re: Performance on shared drive documents
Posted by Karl Wright <da...@gmail.com>.
Hi Ameya,
Windows shared drive access performance is not great, I'm afraid. However,
it is possible that you are overwhelming the server you are talking to.
Try reducing the maximum number of connections for the shared drive
repository connection to between 2 and 5; that sometimes helps.
Karl
On Thu, Jul 24, 2014 at 2:11 PM, Ameya Aware <am...@gmail.com> wrote:
> Hi
>
> When i am crawling full of my C drive the job is running pretty much
> faster.
>
> But at the same time when i m crawling documents kept on shared drive, it
> is taking too much longer time than to crawl C drive.
>
> are documents at shared drive supposed to take to much time??
>
>
> Thanks,
> Ameya
>