You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@manifoldcf.apache.org by kw...@apache.org on 2020/07/07 15:59:31 UTC
svn commit: r1879593 [4/5] - in
/manifoldcf/trunk/site/src/documentation/content/xdocs:
ja_JP/end-user-documentation.xml ja_JP/how-to-build-and-deploy.xml
zh_CN/end-user-documentation.xml zh_CN/how-to-build-and-deploy.xml
Modified: manifoldcf/trunk/site/src/documentation/content/xdocs/zh_CN/end-user-documentation.xml
URL: http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/content/xdocs/zh_CN/end-user-documentation.xml?rev=1879593&r1=1879592&r2=1879593&view=diff
==============================================================================
--- manifoldcf/trunk/site/src/documentation/content/xdocs/zh_CN/end-user-documentation.xml (original)
+++ manifoldcf/trunk/site/src/documentation/content/xdocs/zh_CN/end-user-documentation.xml Tue Jul 7 15:59:31 2020
@@ -1,3202 +1,1364 @@
-<?xml version="1.0"?>
-
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
-<document>
- <header><title>ManifoldCF- End-user Documentation</title></header>
- <properties>
- </properties>
- <body>
-
- <section id="overview">
- <title>Overview</title>
- <p>This manual is intended for an end-user of ManifoldCF. It is assumed that the Framework has been properly installed, either by you or by a system integrator,
- with all required services running and desired connection types properly registered. If you think you need to know how to do that yourself, please visit the "Developer Resources" page.
- </p>
- <p>Most of this manual describes how to use the ManifoldCF user interface. On a standard ManifoldCF deployment, you would reach that interface by giving your browser
- a URL something like this: <code>http://my-server-name:8345/mcf-crawler-ui</code>. This will, of course, differ from system to system. Please contact your system administrator
- to find out what URL is appropriate for your environment.
- </p>
- <p>The ManifoldCF UI has been tested with Firefox and various incarnations of Internet Explorer. If you use another browser, there is a small chance that the UI
- will not work properly. Please let your system integrator know if you find any browser incompatibility problems.</p>
- <p>When you enter the Framework user interface the first time, you will first be asked to log in:</p>
- <br/><br/>
- <figure src="images/en_US/login.PNG" alt="Login Screen" width="80%"/>
- <br/><br/>
- <p>Enter the login user name and password for your system. By default, the user name is "admin" and the password is "admin", although your
- system administrator can (and should) change this. Then, click the "Login" button. If you entered the correct credentials, you should see a
- screen that looks something like this:</p>
- <br/><br/>
- <figure src="images/en_US/welcome-screen.PNG" alt="Welcome Screen" width="80%"/>
- <br/><br/>
- <p>On the left, there are menu options you can select. The main pane on the right shows a welcome message, but depending on what you select on the left, the contents of the main pane
- will change. Before you try to accomplish anything, please take a moment to read the descriptions below of the menu selections, and thus get an idea of how the Framework works
- as a whole.
- </p>
- <section id="outputs">
- <title>Defining Output Connections</title>
- <p>The Framework UI's left-side menu contains a link for listing output connections. An output connection is a connection to a system or place where documents fetched from various
- repositories can be written to. This is often a search engine.</p>
- <p>All jobs must specify an output connection. You can create an output connection by clicking the "List Output Connections" link in the left-side navigation menu. When you do this, the
- following screen will appear:</p>
- <br/><br/>
- <figure src="images/en_US/list-output-connections.PNG" alt="List Output Connections" width="80%"/>
- <br/><br/>
- <p>On a freshly created system, there may well be no existing output connections listed. If there are already output connections, they will be listed on this screen, along with links that allow
- you to view, edit, or delete them. To create a new output connection, click the "Add new output connection" link at the bottom. The following screen will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-output-connection-name.PNG" alt="Add New Output Connection, specify Name" width="80%"/>
- <br/><br/>
- <p>The tabs across the top each present a different view of your output connection. Each tab allows you to edit a different characteristic of that connection. The exact set of tabs you see
- depends on the connection type you choose for the connection.</p>
- <p>Start by giving your connection a name and a description. Remember that all output connection names must be unique, and cannot be changed after the connection is defined. The name must be
- no more than 32 characters long. The description can be up to 255 characters long. When you are done, click on the "Type" tab. The Type tab for the connection will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-output-connection-type.PNG" alt="Add New Output Connection, select Type" width="80%"/>
- <br/><br/>
- <p>The list of output connection types in the pulldown box, and what they are each called, is determined by your system integrator. The configuration tabs for each different kind of output connection
- type are described in separate sections below.</p>
- <p>After you choose an output connection type, click the "Continue" button at the bottom of the pane. You will then see all the tabs appropriate for that kind of connection appear, and a
- "Save" button will also appear at the bottom of the pane. You <b>must</b> click the "Save" button when you are done in order to create your connection. If you click "Cancel" instead, the new connection
- will not be created. (The same thing will happen if you click on any of the navigation links in the left-hand pane.)</p>
- <p>Every output connection has a "Throttling" tab. The tab looks like this:</p>
- <br/><br/>
- <figure src="images/en_US/output-throttling.PNG" alt="Output Connection Throttling" width="80%"/>
- <br/><br/>
- <p>On this tab, you can specify only one thing: how many open connections are allowed at any given time to the system the output connection talks with. This restriction helps prevent
- that system from being overloaded, or in some cases exceeding its license limitations. Conversely, making this number larger allows for greater overall throughput. The default
- value is 10, which may not be optimal for all types of output connections. Please refer to the section of the manual describing your output connection type for more precise
- recommendations.
- </p>
- <p>Please refer to the section of the manual describing your chosen output connection type for a description of the tabs appropriate for that connection type.</p>
-
- <p>After you save your connection, a summary screen will be displayed that describes your connection's configuration. This looks something like this (although the details will differ
- somewhat based on what connection type you chose):</p>
- <br/><br/>
- <figure src="images/en_US/view-output-connection.PNG" alt="View Output Connection" width="80%"/>
- <br/><br/>
- <p>The summary screen contains a line where the connection's status is displayed. If you did everything correctly, the message "Connection working" will be displayed as a status.
- If there was a problem, you will see a connection-type-specific diagnostic message instead. If this happens, you will need to correct the problem, by either fixing your infrastructure,
- or by editing the connection configuration appropriately, before the output connection will work correctly.</p>
- <p>Also note that there are five buttons along the bottom of the display: "Refresh", "Edit", "Delete", "Re-index all associated documents", and "Remove all associated records". We'll
- go into the purpose for each of these buttons in turn.</p>
- <p>The "Refresh" button simply reloads the view page for the output connection, and updates the connection status. Use this button when you have made changes to the external system
- your output connection is connected to that might affect whether the connection will succeed or not.</p>
- <p>The "Edit" button allows you to go back and edit the connection parameters. Use this button if you want to change the connection's characteristics or specifications in any way.</p>
- <p>The "Delete" button allows you to delete the connection. Use this button if you no longer want the connection to remain in the available list of output connections. Note that ManifoldCF
- will not allow you to delete a connection that is being referenced by a job.</p>
- <p>The "Re-index all associated documents" button will nullify the recorded versions of all documents currently indexed via this connection. This is not a button you would use often. Click
- it when you have changed the configuration of whatever system the output connection is describing, and therefore all documents will eventually need to be reindexed.</p>
- <p>The "Remove all associated documents" button will remove from ManifoldCF all knowledge that any indexing has taken place at all to this connection. This is also not a button you would use
- often. Click it when you have removed the entire index that the output connection describes from the target repository.</p>
- </section>
-
- <section id="transformations">
- <title>Defining Transformation Connections</title>
- <p>The Framework UI's left-side menu contains a link for listing transformation connections. A transformation connection is a connection to an engine where documents fetched from various
- repositories can be manipulated. This typically involves metadata extraction or mapping.</p>
- <p>A job does not need to specify any transformation connections. In many cases, the final destination search engine has an included data conversion pipeline. But in the case
- where such data extraction and conversion is not available, ManifoldCF provides a way of taking care of it internally.</p>
- <p>You can create a transformation connection by clicking the "List Transformation Connections" link in the left-side navigation menu. When you do this, the
- following screen will appear:</p>
- <br/><br/>
- <figure src="images/en_US/list-transformation-connections.PNG" alt="List Transformation Connections" width="80%"/>
- <br/><br/>
- <p>On a freshly created system, there may well be no existing transformation connections listed. If there are already transformation connections, they will be listed on this screen, along with links that allow
- you to view, edit, or delete them. To create a new transformation connection, click the "Add new transformation connection" link at the bottom. The following screen will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-transformation-connection-name.PNG" alt="Add New Transformation Connection, specify Name" width="80%"/>
- <br/><br/>
- <p>The tabs across the top each present a different view of your transformation connection. Each tab allows you to edit a different characteristic of that connection. The exact set of tabs you see
- depends on the connection type you choose for the connection.</p>
- <p>Start by giving your connection a name and a description. Remember that all transformation connection names must be unique, and cannot be changed after the connection is defined. The name must be
- no more than 32 characters long. The description can be up to 255 characters long. When you are done, click on the "Type" tab. The Type tab for the connection will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-transformation-connection-type.PNG" alt="Add New Transformation Connection, select Type" width="80%"/>
- <br/><br/>
- <p>The list of transformation connection types in the pulldown box, and what they are each called, is determined by your system integrator. The configuration tabs for each different kind of transformation connection
- type are described in separate sections below.</p>
- <p>After you choose a transformation connection type, click the "Continue" button at the bottom of the pane. You will then see all the tabs appropriate for that kind of connection appear, and a
- "Save" button will also appear at the bottom of the pane. You <b>must</b> click the "Save" button when you are done in order to create your connection. If you click "Cancel" instead, the new connection
- will not be created. (The same thing will happen if you click on any of the navigation links in the left-hand pane.)</p>
- <p>Every transformation connection has a "Throttling" tab. The tab looks like this:</p>
- <br/><br/>
- <figure src="images/en_US/transformation-throttling.PNG" alt="Transformation Connection Throttling" width="80%"/>
- <br/><br/>
- <p>On this tab, you can specify only one thing: how many open connections are allowed at any given time to the system the transformation connection talks with. This restriction helps prevent
- that system from being overloaded, or in some cases exceeding its license limitations. Conversely, making this number larger allows for greater overall throughput. The default
- value is 10, which may not be optimal for all types of output connections. Please refer to the section of the manual describing your transformation connection type for more precise
- recommendations.
- </p>
- <p>Please refer to the section of the manual describing your chosen transformation connection type for a description of the tabs appropriate for that connection type.</p>
-
- <p>After you save your connection, a summary screen will be displayed that describes your connection's configuration. This looks something like this (although the details will differ
- somewhat based on what connection type you chose):</p>
- <br/><br/>
- <figure src="images/en_US/view-transformation-connection.PNG" alt="View Transformation Connection" width="80%"/>
- <br/><br/>
- <p>The summary screen contains a line where the connection's status is displayed. If you did everything correctly, the message "Connection working" will be displayed as a status.
- If there was a problem, you will see a connection-type-specific diagnostic message instead. If this happens, you will need to correct the problem, by either fixing your infrastructure,
- or by editing the connection configuration appropriately, before the transformation connection will work correctly.</p>
- <p>Also note that there are three buttons along the bottom of the display: "Refresh", "Edit", and "Delete". We'll
- go into the purpose for each of these buttons in turn.</p>
- <p>The "Refresh" button simply reloads the view page for the transformation connection, and updates the connection status. Use this button when you have made changes to the external system
- your transformation connection is connected to that might affect whether the connection will succeed or not.</p>
- <p>The "Edit" button allows you to go back and edit the connection parameters. Use this button if you want to change the connection's characteristics or specifications in any way.</p>
- <p>The "Delete" button allows you to delete the connection. Use this button if you no longer want the connection to remain in the available list of transformation connections. Note that ManifoldCF
- will not allow you to delete a connection that is being referenced by a job.</p>
- </section>
-
- <section id="groups">
- <title>Defining Authority Groups</title>
- <p>The Framework UI's left-side menu contains a link for listing authority groups. An authority group is a collection of authorities that all cooperate to furnish
- security for each document from repositories that you select. For example, a SharePoint 2010 repository with the Claims Based authorization feature enabled may
- contain documents that are authorized by SharePoint itself, by Active Directory, and by others. Documents from such a SharePoint
- repository would therefore refer to a authority group which would have a SharePoint native authority, a SharePoint Active Directory authority,
- and other SharePoint claims based authorities as members. But most of the time, an authority group will consist of a single authority that is appropriate for
- the repository the authority group is meant to secure.</p>
- <p>Since you need to select an authority group when you define an authority connection, you should define your authority groups <b>before</b> setting
- up your authority connections. If you don't have any authority groups defined, you cannot create authority connections at all. But if you select the
- wrong authority group when setting up your authority connection, you can go back later and change your selection.</p>
- <p>It is also a good idea to define your authority groups before creating any repository connections, since each repository connection will also need to
- refer back to an authority group in order to secure documents. While it is possible to change the relationship between a repository connection
- and its authority group after-the-fact, in practice such changes may cause many documents to be reindexed the next time an associated job is run.</p>
- <p>You can create an authority group by clicking the "List Authority Groups" link in the left-side navigation menu. When you do this, the
- following screen will appear:</p>
- <br/><br/>
- <figure src="images/en_US/list-authority-groups.PNG" alt="List Authority Groups" width="80%"/>
- <br/><br/>
- <p>If there are already authority groups, they will be listed on this screen, along with links that allow you to view, edit, or delete them. To create a new
- authority group, click the "Add a new authority group" link at the bottom. The following screen will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-authority-group-name.PNG" alt="Add New Authority Group, specify Name" width="80%"/>
- <br/><br/>
- <p>The tabs across the top each present a different view of your authority group. For authority groups, there is only ever one tab, the "Name" tab.</p>
- <p>Give your authority group a name and a description. Remember that all authority group names must be unique, and cannot be changed after the
- authority group is defined. The name must be no more than 32 characters long. The description can be up to 255 characters long. When you are
- done, click on the "Save" button. You <b>must</b> click the "Save" button when you are done in order to create or update your authority group.
- If you click "Cancel" instead, the new authority group will not be created. (The same thing will happen if you click on any of the navigation links in
- the left-hand pane.)</p>
- <p>After you save your authority group, a summary screen will be displayed that describes the group, and you can proceed on to create any authority
- connections that belong to the authority group, or repository connections that refer to the authority group.</p>
-
- </section>
-
- <section id="connections">
- <title>Defining Repository Connections</title>
- <p>The Framework UI's left-hand menu contains a link for listing repository connections. A repository connection is a connection to the repository system that contains the documents
- that you are interested in indexing.</p>
- <p>All jobs require you to specify a repository connection, because that is where they get their documents from. It is therefore necessary to create a repository connection before
- indexing any documents.</p>
- <p>A repository connection also may have an associated authority group. This specified authority group determines the security environment in which documents
- from the repository connection are attached. While it is possible to change the specified authority group for a repository connection after a crawl has been done,
- in practice this will require all documents associated with that repository connection be reindexed in order to be searchable by anyone. Therefore, we recommend
- that you set up your desired authority group before defining your repository connection.</p>
- <p>You can create a repository connection by clicking the "List Repository Connections" link in the left-side navigation menu. When you do this, the
- following screen will appear:</p>
- <br/><br/>
- <figure src="images/en_US/list-repository-connections.PNG" alt="List Repository Connections" width="80%"/>
- <br/><br/>
- <p>On a freshly created system, there may well be no existing repository connections listed. If there are already repository connections, they will be listed on this
- screen, along with links that allow you to view, edit, or delete them. To create a new repository connection, click the "Add a new connection" link at the bottom.
- The following screen will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-repository-connection-name.PNG" alt="Add New Repository Connection, specify Name" width="80%"/>
- <br/><br/>
- <p>The tabs across the top each present a different view of your repository connection. Each tab allows you to edit a different characteristic of that connection. The exact set of tabs you see
- depends on the connection type you choose for the connection.</p>
- <p>Start by giving your connection a name and a description. Remember that all repository connection names must be unique, and cannot be changed after the connection is defined. The name must be
- no more than 32 characters long. The description can be up to 255 characters long. When you are done, click on the "Type" tab. The Type tab for the connection will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-repository-connection-type.PNG" alt="Add New Repository Connection, select Type" width="80%"/>
- <br/><br/>
- <p>The list of repository connection types in the pulldown box, and what they are each called, is determined by your system integrator. The configuration tabs
- for each different kind of repository connection type are described in this document in separate sections below.</p>
- <p>You may also at this point select the authority group to use to secure all documents fetched from this repository with. You do not need to define your
- authority group's authority connections before doing this step, but you will not be able to search for your documents after indexing them until you do.</p>
- <p>After you choose the desired repository connection type and an authority group (if desired), click the "Continue" button at the bottom of the pane. You will
- then see all the tabs appropriate for that kind of connection appear, and a "Save" button will also appear at the bottom of the pane. You <b>must</b> click
- the "Save" button when you are done in order to create or update your connection. If you click "Cancel" instead, the new connection
- will not be created. (The same thing will happen if you click on any of the navigation links in the left-hand pane.)</p>
- <p>Every repository connection has a "Throttling" tab. The tab looks like this:</p>
- <br/><br/>
- <figure src="images/en_US/repository-throttling.PNG" alt="Repository Connection Throttling" width="80%"/>
- <br/><br/>
- <p>On this tab, you can specify two things. The first is how many open connections are allowed at any given time to the system the repository connection talks with. This restriction helps prevent
- that system from being overloaded, or in some cases exceeding its license limitations. Conversely, making this number larger allows for smaller average search latency. The default
- value is 10, which may not be optimal for all types of repository connections. Please refer to the section of the manual describing your authority connection type for more precise
- recommendations. The second specifies how rapidly, on average, the crawler will fetch documents via this connection.
- </p>
- <p>Each connection type has its own notion of "throttling bin". A throttling bin is the name of a resource whose access needs to be throttled. For example, the Web connection type uses a
- document's server name as the throttling bin associated with the document, since (presumably) it will be access to each individual server that will need to be throttled independently.
- </p>
- <p>On the repository connection "Throttling" tab, you can specify an unrestricted number of throttling descriptions. Each throttling description consists of a regular expression that describes
- a family of throttling bins, plus a helpful description, plus an average number of fetches per minute for each of the throttling bins that matches the regular expression. If a given
- throttling bin matches more than one throttling description, the most conservative fetch rate is chosen.</p>
- <p>The simplest regular expression you can use is the empty regular expression. This will match all of the connection's throttle bins, and thus will allow you to specify a default
- throttling policy for the connection. Set the desired average fetch rate, and click the "Add" button. The throttling tab will then appear something like this:</p>
- <br/><br/>
- <figure src="images/en_US/repository-throttling-with-throttle.PNG" alt="Repository Connection Throttling With Throttle" width="80%"/>
- <br/><br/>
- <p>If no throttle descriptions are added, no fetch-rate throttling will be performed.</p>
- <p>Please refer to the section of the manual describing your chosen repository connection type for a description of the tabs appropriate for that connection type.</p>
-
- <p>After you save your connection, a summary screen will be displayed that describes your connection's configuration. This looks something like this (although the details will differ
- somewhat based on what connection type you chose):</p>
- <br/><br/>
- <figure src="images/en_US/view-repository-connection.PNG" alt="View Repository Connection" width="80%"/>
- <br/><br/>
- <p>The summary screen contains a line where the connection's status is displayed. If you did everything correctly, the message "Connection working" will be displayed as a status.
- If there was a problem, you will see a connection-type-specific diagnostic message instead. If this happens, you will need to correct the problem, by either fixing your infrastructure,
- or by editing the connection configuration appropriately, before the repository connection will work correctly.</p>
- <p>Also note that there are four buttons along the bottom of the display: "Refresh", "Edit", "Delete", and "Clear all related history". We'll
- go into the purpose for each of these buttons in turn.</p>
- <p>The "Refresh" button simply reloads the view page for the repository connection, and updates the connection status. Use this button when you have made changes to the external system
- your repository connection is connected to that might affect whether the connection will succeed or not.</p>
- <p>The "Edit" button allows you to go back and edit the connection parameters. Use this button if you want to change the connection's characteristics or specifications in any way.</p>
- <p>The "Delete" button allows you to delete the connection. Use this button if you no longer want the connection to remain in the available list of repository connections. Note that ManifoldCF
- will not allow you to delete a connection that is being referenced by a job.</p>
- <p>The "Clear all related history" button will remove all history data associated with the current repository connection. This is not a button you would use often. History data is used to construct
- reports, such as the "Simple History" report. It is valuable as a diagnostic aid to understand what the crawler has been doing. There is an automated way of configuring ManifoldCF to
- remove history that is older than a specified interval before the current time. But if you want to remove all the history right away, this button will do that.</p>
- </section>
-
- <section id="notifications">
- <title>Defining Notification Connections</title>
- <p>The Framework UI's left-side menu contains a link for listing notification connections. A notification connection is a connection to an engine that generates notification messages, such
- as email or text messages, specifically to note the end or unexpected termination of a job.</p>
- <p>Jobs may specify one or more notification connections. You can create a notification connection by clicking the "List Notification Connections" link in the left-side navigation menu. When you do this, the
- following screen will appear:</p>
- <br/><br/>
- <figure src="images/en_US/list-notification-connections.PNG" alt="List Notification Connections" width="80%"/>
- <br/><br/>
- <p>On a freshly created system, there may well be no existing notification connections listed. If there are already notification connections, they will be listed on this screen, along with links that allow
- you to view, edit, or delete them. To create a new notification connection, click the "Add new notification connection" link at the bottom. The following screen will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-notification-connection-name.PNG" alt="Add New Notification Connection, specify Name" width="80%"/>
- <br/><br/>
- <p>The tabs across the top each present a different view of your notification connection. Each tab allows you to edit a different characteristic of that connection. The exact set of tabs you see
- depends on the connection type you choose for the connection.</p>
- <p>Start by giving your connection a name and a description. Remember that all notification connection names must be unique, and cannot be changed after the connection is defined. The name must be
- no more than 32 characters long. The description can be up to 255 characters long. When you are done, click on the "Type" tab. The Type tab for the connection will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-notification-connection-type.PNG" alt="Add New Notification Connection, select Type" width="80%"/>
- <br/><br/>
- <p>The list of notification connection types in the pulldown box, and what they are each called, is determined by your system integrator. The configuration tabs for each different kind of notification connection
- type are described in separate sections below.</p>
- <p>After you choose a notification connection type, click the "Continue" button at the bottom of the pane. You will then see all the tabs appropriate for that kind of connection appear, and a
- "Save" button will also appear at the bottom of the pane. You <b>must</b> click the "Save" button when you are done in order to create your connection. If you click "Cancel" instead, the new connection
- will not be created. (The same thing will happen if you click on any of the navigation links in the left-hand pane.)</p>
- <p>Every notification connection has a "Throttling" tab. The tab looks like this:</p>
- <br/><br/>
- <figure src="images/en_US/notification-throttling.PNG" alt="Notification Connection Throttling" width="80%"/>
- <br/><br/>
- <p>On this tab, you can specify only one thing: how many open connections are allowed at any given time to the system the notification connection talks with. This restriction helps prevent
- that system from being overloaded, or in some cases exceeding its license limitations. Conversely, making this number larger allows for greater overall throughput. The default
- value is 10, which may not be optimal for all types of notificaiton connections. Please refer to the section of the manual describing your notification connection type for more precise
- recommendations.
- </p>
- <p>Please refer to the section of the manual describing your chosen notification connection type for a description of the tabs appropriate for that connection type.</p>
-
- <p>After you save your connection, a summary screen will be displayed that describes your connection's configuration. This looks something like this (although the details will differ
- somewhat based on what connection type you chose):</p>
- <br/><br/>
- <figure src="images/en_US/view-notification-connection.PNG" alt="View Notification Connection" width="80%"/>
- <br/><br/>
- <p>The summary screen contains a line where the connection's status is displayed. If you did everything correctly, the message "Connection working" will be displayed as a status.
- If there was a problem, you will see a connection-type-specific diagnostic message instead. If this happens, you will need to correct the problem, by either fixing your infrastructure,
- or by editing the connection configuration appropriately, before the output connection will work correctly.</p>
- <p>Also note that there are three buttons along the bottom of the display: "Refresh", "Edit", and "Delete". We'll
- go into the purpose for each of these buttons in turn.</p>
- <p>The "Refresh" button simply reloads the view page for the notification connection, and updates the connection status. Use this button when you have made changes to the external system
- your output connection is connected to that might affect whether the connection will succeed or not.</p>
- <p>The "Edit" button allows you to go back and edit the connection parameters. Use this button if you want to change the connection's characteristics or specifications in any way.</p>
- <p>The "Delete" button allows you to delete the connection. Use this button if you no longer want the connection to remain in the available list of notification connections. Note that ManifoldCF
- will not allow you to delete a connection that is being referenced by a job.</p>
- </section>
-
- <section id="mappers">
- <title>Defining User Mapping Connections</title>
- <p>The Framework UI's left-side menu contains a link for listing user mapping connections. A user mapping connection is a connection to a system
- that understands how to map a user name into a different user name. For example, if you want to enforce document security using LiveLink, but
- you have only an Active Directory user name, you will need to map the Active Directory user name to a corresponding LiveLink one, before finding
- access tokens for it using the LiveLink Authority.</p>
- <p>Not all user mapping connections need to access other systems in order to be useful. ManifoldCF, for instance, comes with a regular expression
- user mapper that manipulates a user name string using regular expressions alone. Also, user mapping is not needed for many, if not most, authorities.
- You will not need any user mapping connections if the authorities that you intend to create can all operate using the same user name, and that user
- name is in the form that will be made available to ManifoldCF's authority servlet at search time.</p>
- <p>You should define your mapping connections <b>before</b> setting up your authority connections. An authority connections may specify a mapping
- connection that precedes it. For the same reason, it's also convenient to define your mapping connections in the order that you want to process the
- user name. If you don't manage to do this right the first time, though, there is no reason you cannot go back and fix things up.</p>
- <p>You can create a mapping connection by clicking the "List User Mapping Connections" link in the left-side navigation menu. When you do this, the
- following screen will appear:</p>
- <br/><br/>
- <figure src="images/en_US/list-mapping-connections.PNG" alt="List User Mapping Connections" width="80%"/>
- <br/><br/>
- <p>On a freshly created system, there may well be no existing mapping connections listed. If there are already mapping connections, they will be listed on this screen, along with links
- that allow you to view, edit, or delete them. To create a new mapping connection, click the "Add a new connection" link at the bottom. The following screen will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-mapping-connection-name.PNG" alt="Add New User Mapping Connection, specify Name" width="80%"/>
- <br/><br/>
- <p>The tabs across the top each present a different view of your mapping connection. Each tab allows you to edit a different characteristic of that connection. The exact set of tabs you see
- depends on the connection type you choose for the connection.</p>
- <p>Start by giving your connection a name and a description. Remember that all mapping connection names must be unique, and cannot be changed after the connection is defined. The name must be
- no more than 32 characters long. The description can be up to 255 characters long. When you are done, click on the "Type" tab. The Type tab for the connection will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-mapping-connection-type.PNG" alt="Add New User Mapping Connection, select Type" width="80%"/>
- <br/><br/>
- <p>The list of mapping connection types in the pulldown box, and what they are each called, is determined by your system integrator. The configuration tabs for each different kind of
- mapping connection type included with ManifoldCF are described in separate sections below.</p>
- <p>After you choose a mapping connection type, click the "Continue" button at the bottom of the pane. You will then see all the tabs appropriate for that kind of connection appear, and a
- "Save" button will also appear at the bottom of the pane. You <b>must</b> click the "Save" button when you are done in order to create your connection. If you click "Cancel" instead,
- the new connection will not be created. (The same thing will happen if you click on any of the navigation links in the left-hand pane.)</p>
- <p>Every mapping connection has a "Prerequisites" tab. This tab allows you to specify which mapping connection needs to be run before this one (if any). The tab looks like this:</p>
- <br/><br/>
- <figure src="images/en_US/mapping-prerequisites.PNG" alt="User Mapping Connection Prerequisites" width="80%"/>
- <br/><br/>
- <p>Note: It is very important that you do not specify prerequisites in such a way as to create a loop. To make this easier, ManifoldCF will not display any user mapping connections in the pulldown
- which, if selected, would lead to a loop.</p>
- <p>Every mapping connection has a "Throttling" tab. The tab looks like this:</p>
- <br/><br/>
- <figure src="images/en_US/mapping-throttling.PNG" alt="User Mapping Connection Throttling" width="80%"/>
- <br/><br/>
- <p>On this tab, you can specify only one thing: how many open connections are allowed at any given time to the system the mapping connection talks with. This restriction helps prevent
- that system from being overloaded, or in some cases exceeding its license limitations. Conversely, making this number larger allows for smaller average search latency. The default
- value is 10, which may not be optimal for all types of mapping connections. Please refer to the section of the manual describing your mapping connection type for more precise
- recommendations.
- </p>
- <p>Please refer to the section of the manual describing your chosen mapping connection type for a description of the tabs appropriate for that connection type.</p>
-
- <p>After you save your connection, a summary screen will be displayed that describes your connection's configuration. This looks something like this (although the details will differ
- somewhat based on what connection type you chose):</p>
- <br/><br/>
- <figure src="images/en_US/view-mapping-connection.PNG" alt="View Mapping Connection" width="80%"/>
- <br/><br/>
- <p>The summary screen contains a line where the connection's status is displayed. If you did everything correctly, the message "Connection working" will be displayed as a status.
- If there was a problem, you will see a connection-type-specific diagnostic message instead. If this happens, you will need to correct the problem, by either fixing your infrastructure,
- or by editing the connection configuration appropriately, before the mapping connection will work correctly.</p>
- <p>Also note that there are three buttons along the bottom of the display: "Refresh", "Edit", and "Delete". We'll
- go into the purpose for each of these buttons in turn.</p>
- <p>The "Refresh" button simply reloads the view page for the mapping connection, and updates the connection status. Use this button when you have made changes to the external system
- your mapping connection is connected to that might affect whether the connection will succeed or not.</p>
- <p>The "Edit" button allows you to go back and edit the connection parameters. Use this button if you want to change the connection's characteristics or specifications in any way.</p>
- <p>The "Delete" button allows you to delete the connection. Use this button if you no longer want the connection to remain in the available list of mapping connections. Note that ManifoldCF
- will not allow you to delete a connection that is being referenced by another mapping connection or an authority connection.</p>
- </section>
-
- <section id="authorities">
- <title>Defining Authority Connections</title>
- <p>The Framework UI's left-side menu contains a link for listing authority connections. An authority connection is a connection to a system that defines a
- particular security environment. For example, if you want to index some documents that are protected by Active Directory, you would need to configure
- an Active Directory authority connection.</p>
- <p>Bear in mind that only specific authority connection types are compatible with a given repository connection type. Read the details of your desired
- repository type in this document in order to understand how it is designed to be used. You may not need an authority if you do not mind that portions
- of all the documents you want to index are visible to everyone. For web, RSS, and Wiki crawling, this might be the situation. Most other repositories
- have some native security mechanism, however.</p>
- <p>You can create an authority connection by clicking the "List Authority Connections" link in the left-side navigation menu. When you do this, the
- following screen will appear:</p>
- <br/><br/>
- <figure src="images/en_US/list-authority-connections.PNG" alt="List Authority Connections" width="80%"/>
- <br/><br/>
- <p>On a freshly created system, there may well be no existing authority connections listed. If there are already authority connections, they will be listed on this screen, along with links
- that allow you to view, edit, or delete them. To create a new authority connection, click the "Add a new connection" link at the bottom. The following screen will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-authority-connection-name.PNG" alt="Add New Authority Connection, specify Name" width="80%"/>
- <br/><br/>
- <p>The tabs across the top each present a different view of your authority connection. Each tab allows you to edit a different characteristic of that connection. The exact set of tabs you see
- depends on the connection type you choose for the connection.</p>
- <p>Start by giving your connection a name and a description. Remember that all authority connection names must be unique, and cannot be changed after the connection is defined. The name must be
- no more than 32 characters long. The description can be up to 255 characters long. When you are done, click on the "Type" tab. The Type tab for the connection will then appear:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-authority-connection-type.PNG" alt="Add New Authority Connection, select Type" width="80%"/>
- <br/><br/>
- <p>The list of authority connection types in the pulldown box, and what they are each called, is determined by your system integrator. The configuration tabs for
- each different kind of authority connection type are described in this document in separate sections below.</p>
- <p>On this tab, you must also select the authority group that the authority connection you are creating belongs to. Select the appropriate authority group from the
- pulldown.</p>
- <p>You also have the option of selecting a non-default authorization domain. An authorization domain describes which of possibly several user identities the
- authority connection is associated with. For example, a single user may have an Active Directory identity, a LiveLink identity, and a FaceBook identity.
- Your authority connection will be appropriate to only one of those identities. The list of specific authorization domains available is determined by your system
- integrator.</p>
- <p>After you choose an authority connection type, the authority group, and optionally the authorization domain, click the "Continue" button at the bottom of the pane.
- You will then see all the tabs appropriate for that kind of connection appear, and a "Save" button will also appear at the bottom of the pane. You <b>must</b>
- click the "Save" button when you are done in order to create your connection. If you click "Cancel" instead, the new connection
- will not be created. (The same thing will happen if you click on any of the navigation links in the left-hand pane.)</p>
- <p>Every authority connection has a "Prerequisites" tab. This tab allows you to specify which mapping connection needs to be run before this one (if any). The tab looks like this:</p>
- <br/><br/>
- <figure src="images/en_US/authority-prerequisites.PNG" alt="Authority Connection Prerequisites" width="80%"/>
- <br/><br/>
- <p>Every authority connection also has a "Throttling" tab. The tab looks like this:</p>
- <br/><br/>
- <figure src="images/en_US/authority-throttling.PNG" alt="Authority Connection Throttling" width="80%"/>
- <br/><br/>
- <p>On this tab, you can specify only one thing: how many open connections are allowed at any given time to the system the authority connection talks with. This restriction helps prevent
- that system from being overloaded, or in some cases exceeding its license limitations. Conversely, making this number larger allows for smaller average search latency. The default
- value is 10, which may not be optimal for all types of authority connections. Please refer to the section of the manual describing your authority connection type for more precise
- recommendations.
- </p>
- <p>Please refer to the section of the manual describing your chosen authority connection type for a description of the tabs appropriate for that connection type.</p>
-
- <p>After you save your connection, a summary screen will be displayed that describes your connection's configuration. This looks something like this (although the details will differ
- somewhat based on what connection type you chose):</p>
- <br/><br/>
- <figure src="images/en_US/view-authority-connection.PNG" alt="View Authority Connection" width="80%"/>
- <br/><br/>
- <p>The summary screen contains a line where the connection's status is displayed. If you did everything correctly, the message "Connection working" will be displayed as a status.
- If there was a problem, you will see a connection-type-specific diagnostic message instead. If this happens, you will need to correct the problem, by either fixing your infrastructure,
- or by editing the connection configuration appropriately, before the authority connection will work correctly.</p>
- <p>Also note that there are three buttons along the bottom of the display: "Refresh", "Edit", and "Delete". We'll
- go into the purpose for each of these buttons in turn.</p>
- <p>The "Refresh" button simply reloads the view page for the authority connection, and updates the connection status. Use this button when you have made changes to the external system
- your authority connection is connected to that might affect whether the connection will succeed or not.</p>
- <p>The "Edit" button allows you to go back and edit the connection parameters. Use this button if you want to change the connection's characteristics or specifications in any way.</p>
- <p>The "Delete" button allows you to delete the connection. Use this button if you no longer want the connection to remain in the available list of authority connections.</p>
- </section>
-
- <section id="jobs">
- <title>Creating Jobs</title>
- <p>A "job" in ManifoldCF is a description of a set of documents. The Framework's job is to fetch this set of documents come from a specific repository connection,
- transform them using zero or more transformation connections, and
- send them to a specific output connection. The repository connection that is associated with the job will determine exactly how this set of documents is described,
- and to some degree how they are indexed. The output connection associated with the job can also affect how each document is indexed, as will any transformation
- connections that are specified.</p>
- <p>Every job is expected to be run more than once. Each time a job is run, it is responsible not only for sending new or changed documents to the output connection,
- but also for notifying the output connection of any documents that are no longer part of the set. Note that there are two ways for a document to no longer be part
- of the included set of documents: Either the document may have been deleted from the repository, or the document may no longer be included in the allowed set
- of documents. The Framework handles each case properly.</p>
- <p>Deleting a job causes the output connection to be notified of deletion for all documents belonging to that job. This makes sense because the job represents the set
- of documents, which would otherwise be orphaned when the job was removed. (Some users make the assumption that a ManifoldCF job represents nothing more
- than a task, which is an incorrect assumption.)</p>
- <p>Note that the Framework allows jobs that describe overlapping sets of documents to be defined. Documents that exist in more than one job are treated in the
- following special ways:</p>
- <ul>
- <li>When a job is deleted, the output connections are notified of deletion of documents belonging to that job only if they don't belong to another job</li>
- <li>The version of the document sent to an output connection depends on which job was run last</li>
- </ul>
- <p>The subtle logic of overlapping documents means that you probably want to avoid this situation entirely, if it is at all feasible.</p>
- <p>A typical non-continuous run of a job has the following stages of execution:</p>
- <ol>
- <li>Adding the job's new, changed, or deleted starting points to the queue ("seeding")</li>
- <li>Fetching documents, discovering new documents, and detecting deletions</li>
- <li>Removing no-longer-included documents from the queue</li>
- </ol>
- <p>Jobs can also be run "continuously", which means that the job never completes, unless it is aborted. A continuous run has different stages of execution:</p>
- <ol>
- <li>Adding the job's new, changed, or deleted starting points to the queue ("seeding")</li>
- <li>Fetching documents, discovering new documents, and detecting deletions, while reseeding periodically</li>
- </ol>
- <p>Note that continuous jobs <b>cannot</b> remove no-longer-included documents from the queue. They can only remove documents that have been deleted from
- the repository.</p>
- <p>A job can independently be configured to start when explicitly started by a user, or to run on a user-specified schedule. If a job is set up to run on a schedule, it
- can be made to start only at the beginning of a schedule window, or to start again within any remaining schedule window when the previous job run completes.</p>
- <p>There is no restriction in ManifoldCF as to how many jobs many running at any given time.</p>
- <p>You create a job by first clicking on the "List All Jobs" link on the left-side menu. The following screen will appear:</p>
- <br/><br/>
- <figure src="images/en_US/list-jobs.PNG" alt="List Jobs" width="80%"/>
- <br/><br/>
- <p>You may view, edit, or delete any existing jobs by clicking on the appropriate link. You may also create a new job that is a copy of an existing job. But to create
- a brand-new job, click the "Add a new job" link at the bottom. You will then see the following page:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-job-name.PNG" alt="Add New Job, name tab" width="80%"/>
- <br/><br/>
- <p>Give your job a name. Note that job names do <b>not</b> have to be unique, although it is probably less confusing to have a different name for each one. Then,
- click the "Connection" tab:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-job-connection.PNG" alt="Add New Job, connection tab" width="80%"/>
- <br/><br/>
- <p>Now, you should select the repository connection name. Bear in mind that whatever you select cannot be changed after the
- job is saved the first time.</p>
- <p>Add an output, or more than one, by selecting the output in the pulldown, selecting the prerequisite pipeline stage, and clicking the "Add output" button.
- Note that once the job is saved the first time, you cannot delete an output. But you can rearrange your document processing pipeline in most other ways whenever
- you want to, including adding or removing transformation connections.</p>
- <p>If you do not have any transformation connections defined, you will not be given the option of inserting a transformation connection into the pipeline. But if
- you have transformation connections defined, and you want to include them in the document pipeline, you can select them from the transformation connection
- pulldown, type a description into the description box, and then click one of the "Insert before" buttons to insert it into the document pipeline.</p>
- <p>If you do not have any notification connections defined, you will not be given the option of adding one or more notifications to the end of the job. But if
- you have notification connections defined, and you want to include them, you can select them from the notification connection
- pulldown, type a description into the description box, and then click the appropriate "Add" button to add it into the notification list.</p>
- <p>You also have the opportunity to modify the job's priority and start method at this time. The priority
- controls how important this job's documents are, relative to documents from any other job. The higher the number, the more important it is considered for that job's
- documents to be fetched first. The start method is as previously described; you get a choice of manual start, starting on the beginning of a scheduling window, or
- starting whenever possible within a scheduling window.</p>
- <p>Make your selections, and click "Continue". The rest of the job's tabs will now appear, and a
- "Save" button will also appear at the bottom of the pane. You <b>must</b> click the "Save" button when you are done in order to create or update your job. If
- you click "Cancel" instead, the new job will not be created. (The same thing will happen if you click on any of the navigation links in the left-hand pane.)</p>
- <p>All jobs have a "Scheduling" tab. The scheduling tab allows you to set up schedule-related configuration information:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-job-scheduling.PNG" alt="Add New Job, scheduling tab" width="80%"/>
- <br/><br/>
- <p>On this tab, you can specify the following parameters:</p>
- <ul>
- <li>Whether the job runs continuously, or scans every document once</li>
- <li>How long a document should remain alive before it is 'expired', and removed from the index</li>
- <li>The minimum interval before a document is re-checked, to see if it has changed</li>
- <li>The maximum interval before a document is re-checked, to see if it has changed</li>
- <li>How long to wait before reseeding initial documents</li>
- </ul>
- <br/>
- <p>The last four parameters only make sense if a job is a continuously running one, as the UI indicates.</p>
- <p>The other thing you can do on this time is to define an appropriate set of scheduling records. Each scheduling record defines some related set of intervals during
- which the job can run. The intervals are determined by the starting time (which is defined by the day of week, month, day, hour, and minute pulldowns), and the
- maximum run time in minutes, which determines when the interval ends. It is, of course, possible to select multiple values for each of the pulldowns, in which case
- you be describing a starting time that had to match at least <b>one</b> of the selected values for <b>each</b> of the specified fields.</p>
- <p>Once you have selected the schedule values you want, click the "Add Scheduled Time" button:</p>
- <br/><br/>
- <figure src="images/en_US/add-new-job-scheduling-with-record.PNG" alt="Add New Job, scheduling tab with record" width="80%"/>
- <br/><br/>
- <p>The example shows a schedule where crawls are run on Saturday and Sunday nights at 2 AM, and run for no more than 4 hours.</p>
- <p>The rest of the job tabs depend on the types of the connections you selected. Please refer to the section of the manual
- describing the appropriate connection types corresponding to your chosen repository and output connections for a description of the job tabs that will appear for
- those connections.</p>
-
- <p>After you save your job, a summary screen will be displayed that describes your job's specification. This looks something like this (although the details will differ
- somewhat based on what connections you chose):</p>
- <br/><br/>
- <figure src="images/en_US/view-job.PNG" alt="View Job" width="80%"/>
- <br/><br/>
- <p>Also note that there are four buttons along the bottom of the display: "Edit", "Delete", "Copy", and "Reset seeding". We'll
- go into the purpose for each of these buttons in turn.</p>
- <p>The "Edit" button allows you to go back and edit the job specification. Use this button if you want to change the job's details in any way.</p>
- <p>The "Delete" button allows you to delete the job. Use this button if you no longer want the job to exist. Note that when you delete a job in ManifoldCF, all
- documents that were indexed using that job are removed from the index.</p>
- <p>The "Copy" button allows you to edit a copy of the current job. Use this button if you want to create a new job that is based largely on the current job's specification.
- This can be helpful if you have many similar jobs to create.</p>
- <p>The "Reset seeding" button will cause ManifoldCF to forget the seeding history of the job. Seeding is the process of discovering documents that have been added
- or modified. Clicking this button insures that ManifoldCF will examine all documents in the repository on the next crawl. This is not something that is done frequently;
- ManifoldCF is pretty good at managing this information itself, and will automatically do the same thing whenever a job specification is changed. Use this option if
- you've updated your connector software in a way that requires all documents to be re-examined.</p>
- </section>
-
- <section id="executing">
- <title>Executing Jobs</title>
- <p>You can follow what is going on, and control the execution of your jobs, by clicking on the "Status and Job Management" link on the left-side navigation menu. When you do, you might
- see something like this:</p>
- <br/><br/>
- <figure src="images/en_US/job-status.PNG" alt="Job Status" width="80%"/>
- <br/><br/>
- <p>From here, you can click the "Refresh" link at the bottom of the main pane to see an updated status display, or you can directly control the job using the links in the leftmost
- status column. Allowed actions you may see at one point or another include:</p>
- <ul>
- <li>Start (start the job)</li>
- <li>Start minimal (start the job, but do only the minimal work possible)</li>
- <li>Abort(abort the job)</li>
- <li>Pause (pause the job)</li>
- <li>Resume (resume the job)</li>
- <li>Restart (equivalent to aborting the job, and starting it all over again)</li>
- <li>Restart minimal (equivalent to aborting the job, and starting it all over again, doing only the minimal work possible)</li>
- </ul>
- <br/>
- <p>The columns "Documents", "Active", and "Processed" have very specific means as far as documents in the job's queue are concerned. The "Documents" column counts all the documents
- that belong to the job. The "Active" column counts all of the documents for that job that are queued up for processing. The "Processed" column counts all documents that are on the
- queue for the job that have been processed at least once in the past.</p>
- <p>Using the "minimal" variant of the listed actions will perform the minimum possible amount of work, given the model that the connection type for the job uses. In some
- cases, this will mean that additions and modifications are indexed, but deletions are not detected. A complete job run is usually necessary to fully synchronize the target
- index with the repository contents.</p>
- </section>
- <section id="statusreports">
- <title>Status Reports</title>
- <p>Every job in ManifoldCF describes a set of documents. A reference to each document in the set is kept in a job-specific queue. It is sometimes valuable for
- diagnostic reasons to examine this queue for information. The Framework UI has several canned reports which do just that.</p>
- <p>Each status report allows you to select what documents you are interested in from a job's queue based on the following information:</p>
- <ul>
- <li>The job</li>
- <li>The document identifier</li>
- <li>The document's status and state</li>
- <li>When the document is scheduled to be processed next</li>
- </ul>
- <section id="documentstatus">
- <title>Document Status</title>
- <p>A document status report simply lists all matching documents from within the queue, along with their state, status, and planned future activity. You might use this report if you were
- trying to figure out (for example) whether a specific document had been processed yet during a job run.</p>
- <p>Click on the "Document Status" link on the left-hand menu. You will see a screen that looks something like this:</p>
- <br/><br/>
- <figure src="images/en_US/document-status-select-connection.PNG" alt="Document Status, select connection" width="80%"/>
- <br/><br/>
- <p>Select the desired connection. You may also select the desired document state and status, as well as specify a regular expression for the document identifier, if you want. Then,
- click the "Continue" button:</p>
- <br/><br/>
- <figure src="images/en_US/document-status-select-job.PNG" alt="Document Status, select job" width="80%"/>
- <br/><br/>
- <p>Select the job whose documents you want to see, and click "Continue" once again. The results will display:</p>
- <br/><br/>
- <figure src="images/en_US/document-status-example.PNG" alt="Document Status, example" width="80%"/>
- <br/><br/>
- <p>You may alter the criteria, and click "Go" again, if you so choose. Or, you can alter the number of result rows displayed at a time, and click "Go" to redisplay. Finally, you can page
- up and down through the results using the "Prev" and "Next" links.</p>
- </section>
- <section id="queuestatus">
- <title>Queue Status</title>
- <p>A queue status report is an aggregate report that counts the number of occurrences of documents in specified classes. The classes are specified as a grouping within a regular
- expression, which is matched against all specified document identifiers. The results that are displayed are counts of documents. There will be a column for each combination of
- document state and status.</p>
- <p>For example, a class specification of "()" will produce exactly one result row, and will provide a count of documents that are in each state/status combination. A class description
- of "(.*)", on the other hand, will create one row for each document identifier, and will put a "1" in the column representing state and status of that document, with a "0" in all other
- column positions.</p>
- <p>Click the "Queue Status" link on the left-hand menu. You will see a screen that looks like this:</p>
- <br/><br/>
- <figure src="images/en_US/queue-status-select-connection.PNG" alt="Queue Status, select connection" width="80%"/>
- <br/><br/>
- <p>Select the desired connection. You may also select the desired document state and status, as well as specify a regular expression for the document identifier, if you want. You
- will probably want to change the document identifier class from its default value of "(.*)". Then, click the "Continue" button:</p>
- <br/><br/>
- <figure src="images/en_US/queue-status-select-job.PNG" alt="Queue Status, select job" width="80%"/>
- <br/><br/>
- <p>Select the job whose documents you want to see, and click "Continue" once again. The results will display:</p>
- <br/><br/>
- <figure src="images/en_US/queue-status-example.PNG" alt="Queue Status, example" width="80%"/>
- <br/><br/>
- <p>You may alter the criteria, and click "Go" again, if you so choose. Or, you can alter the number of result rows displayed at a time, and click "Go" to redisplay. Finally, you can page
- up and down through the results using the "Prev" and "Next" links.</p>
-
- </section>
- </section>
- <section id="historyreports">
- <title>History Reports</title>
- <p>For every repository connection, ManifoldCF keeps a history of what has taken place involving that connection. This history includes both events that the
- framework itself logs, as well as events that a repository connection or output connection will log. These individual events are categorized by "activity type". Some of the kinds of
- activity types that exist are:</p>
- <ul>
- <li>Job start</li>
- <li>Job end</li>
- <li>Job abort</li>
- <li>Various connection-type-specific read or access operations</li>
- <li>Various connection-type-specific output or indexing operations</li>
- </ul>
- <p>This history can be enormously helpful in understand how your system is behaving, and whether or not it is working properly. For this reason, the Framework UI has the ability to
- generate several canned reports which query this history data and display the results.</p>
- <p>All history reports allow you to specify what history records you are interested in including. These records are selected using the following criteria:</p>
- <ul>
- <li>The repository connection name</li>
- <li>The activity type(s) desired</li>
- <li>The start time desired</li>
- <li>The end time desired</li>
- <li>The identifier(s) involved, specified as a regular expression</li>
- <li>The result(s) produced, specified as a regular expression</li>
- </ul>
- <p>The actual reports available are designed to be useful for diagnosing both access issues, and performance issues. See below for a summary of the types available.</p>
- <section id="simplehistory">
- <title>Simple History Reports</title>
- <p>As the name suggests, a simple history report does not attempt to aggregate any data, but instead just lists matching records from the repository connection's history.
- These records are initially presented in most-recent-first order, and include columns for the start and end time of the event, the kind of activity represented by the event,
- the identifier involved, the number of bytes involved, and the results of the event. Once displayed, you may choose to display more or less data, or reorder the display by column, or page through the data.</p>
- <p>To get started, click on the "Simple History" link on the left-hand menu. You will see a screen that looks like this:</p>
- <br/><br/>
- <figure src="images/en_US/simple-history-select-connection.PNG" alt="Simple History Report, select connection" width="80%"/>
- <br/><br/>
- <p>Now, select the desired repository connection from the pulldown in the upper left hand corner. If you like, you can also change the specified date/time range, or specify an identifier
- regular expression or result code regular expression. By default, the date/time range selects all events within the last hour, while the identifier regular expression and result code
- regular expression matches all identifiers and result codes.</p>
- <p>Next, click the "Continue" button. A list of pertinent activities should then appear in a pulldown in the upper right:</p>
- <br/><br/>
- <figure src="images/en_US/simple-history-select-activities.PNG" alt="Simple History Report, select activities" width="80%"/>
- <br/><br/>
- <p>You may select one or more activities that you would like a report on. When you are done, click the "Go" button. The results will appear, ordered by time, most recent event first:</p>
- <br/><br/>
- <figure src="images/en_US/simple-history-example.PNG" alt="Simple History Report, example" width="80%"/>
- <br/><br/>
- <p>You may alter the criteria, and click "Go" again, if you so choose. Or, you can alter the number of result rows displayed at a time, and click "Go" to redisplay. Finally, you can page
- up and down through the results using the "Prev" and "Next" links.</p>
- <p>Please bear in mind that the report redisplays whatever matches each time you click "Go". So, if your time interval goes from an hour beforehand to "now", and you have activity
- happening, you will see different results each time "Go" is clicked.</p>
- </section>
- <section id="maxactivity">
- <title>Maximum Activity Reports</title>
- <p>A maximum activity report is an aggregate report used primarily to display the maximum rate that events occur within a specified time interval. MHL</p>
- </section>
- <section id="maxbandwidth">
- <title>Maximum Bandwidth Reports</title>
- <p>A maximum bandwidth report is an aggregate report used primarily to display the maximum byte rate that pertains to events occurring within a specified time interval. MHL</p>
- </section>
- <section id="resulthistogram">
- <title>Result Histogram Reports</title>
- <p>A result histogram report is an aggregate report is used to count the occurrences of each kind of matching result for all matching events. MHL</p>
- </section>
- </section>
-
- <section id="credentials">
- <title>A Note About Credentials</title>
- <p>If any of your selected connection types require credentials, you may find it necessary to approach your system administrator to obtain an appropriate set. System administrators
- are often reluctant to provide accounts and credentials that have any more power than is utterly necessary, and sometimes not even that. Great care has been taken in the
- development of all connection types to be sure they require no more privilege than is utterly necessary. If a security-related warning appears when you view a connection's
- status, you must inform the system administrator that the credentials are inadequate to allow the connection to accomplish its task, and work with him/her to correct the problem.
- </p>
- </section>
-
- </section>
-
- <section id="outputconnectiontypes">
- <title>Output Connection Types</title>
-
- <section id="amazoncloudsearchoutputconnector">
- <title>Amazon Cloud Search Output Connection</title>
- <p>The Amazon Cloud Search output connection type send documents to a specific path within a specified Amazon Cloud Search instance. The
- connection type furthermore "batches" documents to reduce cost as much as is reasonable. As a result, some specified documents may be sent at the
- end of a job run, rather than at the time they would typically be indexed.</p>
- <p>The connection configuration information for the Amazon Cloud Search Output Connection type includes one additional tab: the "Server" tab.
- This tab looks like this:</p>
- <br/><br/>
- <figure src="images/en_US/amazon-configure-server.PNG" alt="Amazon Output Configuration, Server tab" width="80%"/>
- <br/><br/>
- <p>You must supply the "Server host" field in order for the connection to work.</p>
- <p>The Amazon Cloud Search Output Connection type does not contribute any tabs to a job definition.</p>
- <p>The Amazon Cloud Search Output Connection type can only accept text content that is encoded in a UTF-8-compatible manner. It is highly
- recommended to use the Tika Content Extractor in the pipeline prior to the Amazon Cloud Search Output Connection type in order to
- convert documents to an indexable form.</p>
- <p></p>
- <p>In order to successfully index ManifoldCF documents in Amazon Cloud Search, you will need to describe a Cloud Search schema for
- receiving them. The fields that the Amazon Cloud Search output connection type sends are those that it gets specifically from the document
- as it comes through the ManifoldCF pipeline, with the addition of two hard-wired fields: "f_bodytext", containing the document body content,
- and "document_uri", containing the document's URI. You may also need to use the Metadata Adjuster transformation connection type to make
- sure that document metadata sent to Amazon Cloud Search agree with the schema you have defined there. Please refer to
- <a href="http://docs.aws.amazon.com/cloudsearch/latest/developerguide/configuring-index-fields.html">this document</a> for details of how to set up an Amazon Cloud Search
- schema.</p>
- </section>
-
- <section id="cmisoutput">
- <title>CMIS Output Connection</title>
- <p>The CMIS Output Connection type allows you to migrate content to any CMIS-compliant repository.</p>
- <p>By default each CMIS Connection manages a single CMIS repository, this means that if you have multiple CMIS repositories exposed by a single
- endpoint, you need to create a specific connection for each CMIS repository.</p>
- <p>CMIS repository documents are typically secured by using the CMIS Authority Connection type. This authority type, however, does not have access
- to user groups, since there is no such functionality in the CMIS specification at this time. As a result, most people only use the CMIS connection type
- in an unsecured manner.</p>
- <br/>
- <p>A CMIS Output connection has the following configuration parameters on the output connection editing screen:</p>
- <br/><br/>
- <figure src="images/en_US/cmis-output-connection-configuration.png" alt="CMIS Output Connection, configuration parameters" width="80%"/>
- <br/><br/>
- <p>Select the correct CMIS binding protocol (AtomPub or Web Services) and enter the correct username, password and the endpoint to reference the CMIS document server services.</p>
- <p>The endpoint consists of the HTTP protocol, hostname, port and the context path of the CMIS service exposed by the CMIS server:</p>
- <br/>
- <p><code>http://HOSTNAME:PORT/CMIS_CONTEXT_PATH</code></p>
- <br/><br/>
- <p>The CMIS Query must be provided to select your own drop zone in the target folder that should be an existent CMIS folder.</p>
- <p>By default the crawler will replicate the same source path structure for each content in that target folder.</p>
- <p>Considering to have your contents in your source repository inside the following path:</p>
- <br/>
- <p><code>/MySourceRepo/Invoices</code></p>
- <br/>
- <p>And supposing to have configured your CMIS Output Connection with the default value of the CMIS Query for your target folder:</p>
- <br/>
- <p><code>SELECT * FROM cmis:folder WHERE cmis:name='Apache ManifoldCF'</code></p>
- <br /><br/>
- <p>All the migrated contents will be dropped in the following target CMIS folder:</p>
- <br/>
- <p><code>/path/to/your/Apache ManifoldCF/MySourceRepo/Invoices</code></p>
- <br/>
- <p>Optionally you can provide the repository ID to select one of the exposed CMIS repository, if this parameter is null the CMIS Connector will consider the first CMIS repository exposed by the CMIS server.</p>
- <br/>
- <p>Note that, in a CMIS system, a specific binding protocol has its own context path, this means that the endpoints are different:</p>
- <p>for example the endpoint of the AtomPub binding exposed by the actual version of the InMemory Server provided by the OpenCMIS framework is the following:</p>
- <p><code>http://localhost:8080/chemistry-opencmis-server-inmemory-war-0.5.0-SNAPSHOT/atom</code></p>
[... 3761 lines stripped ...]