You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Sami Siren (JIRA)" <ji...@apache.org> on 2005/04/06 21:17:12 UTC
[jira] Created: (NUTCH-38) distributed search improvement
distributed search improvement
------------------------------
Key: NUTCH-38
URL: http://issues.apache.org/jira/browse/NUTCH-38
Project: Nutch
Type: Improvement
Components: searcher, web gui
Reporter: Sami Siren
Assigned to: Sami Siren
Priority: Minor
Attachments: fault_tolerance.patch
Running nutch web application with separate search servers can be pain to manage. Search servers go down, new segments are added, old ones removes and so on, almost all of these actions require some work to be done also at web app level (restart usually helps).
This simple enhancement tries to makes it just a little bit easier adding following features:
-new segments can be added to searchservers without restarting the frontend.
-defective search servers are not queried until tey come back online
-watchdog keeps an eye for your searchservers and writes a simple log abt statistics
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
[jira] Resolved: (NUTCH-38) distributed search improvement
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-38?page=history ]
Sami Siren resolved NUTCH-38:
-----------------------------
Resolution: Fixed
this is now committed
> distributed search improvement
> ------------------------------
>
> Key: NUTCH-38
> URL: http://issues.apache.org/jira/browse/NUTCH-38
> Project: Nutch
> Type: Improvement
> Components: searcher, web gui
> Reporter: Sami Siren
> Assignee: Sami Siren
> Priority: Minor
> Attachments: fault_tolerance.patch, fault_tolerance_b.patch, fault_tolerance_c.patch
>
> Running nutch web application with separate search servers can be pain to manage. Search servers go down, new segments are added, old ones removes and so on, almost all of these actions require some work to be done also at web app level (restart usually helps).
> This simple enhancement tries to makes it just a little bit easier adding following features:
> -new segments can be added to searchservers without restarting the frontend.
> -defective search servers are not queried until tey come back online
> -watchdog keeps an eye for your searchservers and writes a simple log abt statistics
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
[jira] Updated: (NUTCH-38) distributed search improvement
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-38?page=history ]
Sami Siren updated NUTCH-38:
----------------------------
Attachment: fault_tolerance.patch
> distributed search improvement
> ------------------------------
>
> Key: NUTCH-38
> URL: http://issues.apache.org/jira/browse/NUTCH-38
> Project: Nutch
> Type: Improvement
> Components: searcher, web gui
> Reporter: Sami Siren
> Assignee: Sami Siren
> Priority: Minor
> Attachments: fault_tolerance.patch
>
> Running nutch web application with separate search servers can be pain to manage. Search servers go down, new segments are added, old ones removes and so on, almost all of these actions require some work to be done also at web app level (restart usually helps).
> This simple enhancement tries to makes it just a little bit easier adding following features:
> -new segments can be added to searchservers without restarting the frontend.
> -defective search servers are not queried until tey come back online
> -watchdog keeps an eye for your searchservers and writes a simple log abt statistics
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
[jira] Updated: (NUTCH-38) distributed search improvement
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-38?page=history ]
Sami Siren updated NUTCH-38:
----------------------------
Attachment: fault_tolerance_c.patch
please ignore the previous patch, it contained something that should not have been there.
> distributed search improvement
> ------------------------------
>
> Key: NUTCH-38
> URL: http://issues.apache.org/jira/browse/NUTCH-38
> Project: Nutch
> Type: Improvement
> Components: searcher, web gui
> Reporter: Sami Siren
> Assignee: Sami Siren
> Priority: Minor
> Attachments: fault_tolerance.patch, fault_tolerance_b.patch, fault_tolerance_c.patch
>
> Running nutch web application with separate search servers can be pain to manage. Search servers go down, new segments are added, old ones removes and so on, almost all of these actions require some work to be done also at web app level (restart usually helps).
> This simple enhancement tries to makes it just a little bit easier adding following features:
> -new segments can be added to searchservers without restarting the frontend.
> -defective search servers are not queried until tey come back online
> -watchdog keeps an eye for your searchservers and writes a simple log abt statistics
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-38) distributed search improvement
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-38?page=comments#action_62302 ]
Doug Cutting commented on NUTCH-38:
-----------------------------------
This will make a great improvement!
A few comments:
- Your setStop() method should instead override ipc.Client.stop(), and call super.stop(). IPC clients should already call stop(), and that same call should kill the watchdog thread too.
- Instead of iterating through all values of a Vector to convert it to an array, use the toArray method.
- Your field and variable names do not use java's usual conventions, for example, stat_servers should be named statServers.
Thanks!
> distributed search improvement
> ------------------------------
>
> Key: NUTCH-38
> URL: http://issues.apache.org/jira/browse/NUTCH-38
> Project: Nutch
> Type: Improvement
> Components: searcher, web gui
> Reporter: Sami Siren
> Assignee: Sami Siren
> Priority: Minor
> Attachments: fault_tolerance.patch
>
> Running nutch web application with separate search servers can be pain to manage. Search servers go down, new segments are added, old ones removes and so on, almost all of these actions require some work to be done also at web app level (restart usually helps).
> This simple enhancement tries to makes it just a little bit easier adding following features:
> -new segments can be added to searchservers without restarting the frontend.
> -defective search servers are not queried until tey come back online
> -watchdog keeps an eye for your searchservers and writes a simple log abt statistics
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
[jira] Updated: (NUTCH-38) distributed search improvement
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-38?page=history ]
Sami Siren updated NUTCH-38:
----------------------------
Attachment: fault_tolerance_b.patch
Changed according to Dougs comments
> distributed search improvement
> ------------------------------
>
> Key: NUTCH-38
> URL: http://issues.apache.org/jira/browse/NUTCH-38
> Project: Nutch
> Type: Improvement
> Components: searcher, web gui
> Reporter: Sami Siren
> Assignee: Sami Siren
> Priority: Minor
> Attachments: fault_tolerance.patch, fault_tolerance_b.patch
>
> Running nutch web application with separate search servers can be pain to manage. Search servers go down, new segments are added, old ones removes and so on, almost all of these actions require some work to be done also at web app level (restart usually helps).
> This simple enhancement tries to makes it just a little bit easier adding following features:
> -new segments can be added to searchservers without restarting the frontend.
> -defective search servers are not queried until tey come back online
> -watchdog keeps an eye for your searchservers and writes a simple log abt statistics
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-38) distributed search improvement
Posted by "Stefan Grroschupf (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-38?page=comments#action_62382 ]
Stefan Grroschupf commented on NUTCH-38:
----------------------------------------
This a very very great improvement - thanks a lot.
I would be very happy to see it in the sources asap! :-)
> distributed search improvement
> ------------------------------
>
> Key: NUTCH-38
> URL: http://issues.apache.org/jira/browse/NUTCH-38
> Project: Nutch
> Type: Improvement
> Components: searcher, web gui
> Reporter: Sami Siren
> Assignee: Sami Siren
> Priority: Minor
> Attachments: fault_tolerance.patch, fault_tolerance_b.patch, fault_tolerance_c.patch
>
> Running nutch web application with separate search servers can be pain to manage. Search servers go down, new segments are added, old ones removes and so on, almost all of these actions require some work to be done also at web app level (restart usually helps).
> This simple enhancement tries to makes it just a little bit easier adding following features:
> -new segments can be added to searchservers without restarting the frontend.
> -defective search servers are not queried until tey come back online
> -watchdog keeps an eye for your searchservers and writes a simple log abt statistics
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
[jira] Closed: (NUTCH-38) distributed search improvement
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-38?page=all ]
Sami Siren closed NUTCH-38:
---------------------------
> distributed search improvement
> ------------------------------
>
> Key: NUTCH-38
> URL: http://issues.apache.org/jira/browse/NUTCH-38
> Project: Nutch
> Type: Improvement
> Components: searcher, web gui
> Reporter: Sami Siren
> Assignee: Sami Siren
> Priority: Minor
> Attachments: fault_tolerance.patch, fault_tolerance_b.patch, fault_tolerance_c.patch
>
> Running nutch web application with separate search servers can be pain to manage. Search servers go down, new segments are added, old ones removes and so on, almost all of these actions require some work to be done also at web app level (restart usually helps).
> This simple enhancement tries to makes it just a little bit easier adding following features:
> -new segments can be added to searchservers without restarting the frontend.
> -defective search servers are not queried until tey come back online
> -watchdog keeps an eye for your searchservers and writes a simple log abt statistics
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-38) distributed search improvement
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-38?page=comments#action_62309 ]
Doug Cutting commented on NUTCH-38:
-----------------------------------
Looks good. +1
One more minor improvement you might make: statServers and statSegments don't really need to be fields. Rather they can just be variables in the updateSegments() method, and this method can log them directly, rather than the run() method logging them.
> distributed search improvement
> ------------------------------
>
> Key: NUTCH-38
> URL: http://issues.apache.org/jira/browse/NUTCH-38
> Project: Nutch
> Type: Improvement
> Components: searcher, web gui
> Reporter: Sami Siren
> Assignee: Sami Siren
> Priority: Minor
> Attachments: fault_tolerance.patch, fault_tolerance_b.patch, fault_tolerance_c.patch
>
> Running nutch web application with separate search servers can be pain to manage. Search servers go down, new segments are added, old ones removes and so on, almost all of these actions require some work to be done also at web app level (restart usually helps).
> This simple enhancement tries to makes it just a little bit easier adding following features:
> -new segments can be added to searchservers without restarting the frontend.
> -defective search servers are not queried until tey come back online
> -watchdog keeps an eye for your searchservers and writes a simple log abt statistics
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira