You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@synapse.apache.org by hi...@apache.org on 2013/08/19 00:59:22 UTC
svn commit: r1515239 -
/synapse/trunk/java/modules/documentation/src/site/xdoc/userguide/transports/vfs.xml
Author: hiranya
Date: Sun Aug 18 22:59:22 2013
New Revision: 1515239
URL: http://svn.apache.org/r1515239
Log:
Updated documentation
Modified:
synapse/trunk/java/modules/documentation/src/site/xdoc/userguide/transports/vfs.xml
Modified: synapse/trunk/java/modules/documentation/src/site/xdoc/userguide/transports/vfs.xml
URL: http://svn.apache.org/viewvc/synapse/trunk/java/modules/documentation/src/site/xdoc/userguide/transports/vfs.xml?rev=1515239&r1=1515238&r2=1515239&view=diff
==============================================================================
--- synapse/trunk/java/modules/documentation/src/site/xdoc/userguide/transports/vfs.xml (original)
+++ synapse/trunk/java/modules/documentation/src/site/xdoc/userguide/transports/vfs.xml Sun Aug 18 22:59:22 2013
@@ -33,7 +33,17 @@
<a href="#Configuration">Transport Configuration</a>
<ul>
<li><a href="#Listener">File Transport Listener (VFS Listener)</a></li>
- <li><a href="#Sender">File Transport Sender (VFS Sender)</a></li>
+ <li>
+ <a href="#Sender">File Transport Sender (VFS Sender)</a>
+ <ul>
+ <li><a href="#Locking">File Locking</a></li>
+ <li><a href="#Passive">FTP Passive Mode</a></li>
+ <li><a href="#Retry">Retrying on Error</a></li>
+ <li><a href="#transport.vfs.UseTempFile">Using Temporary Files</a></li>
+ <li><a href="#Append">Appending to Files</a></li>
+ <li><a href="#OutOnly">Out-only Message Exchange Pattern</a></li>
+ </ul>
+ </li>
</ul>
</li>
<li>
@@ -137,7 +147,7 @@
a directory location is specified, the transport will attempt to read
any file dropped into the directory.
<div class="xmlConf"><parameter name="transport.vfs.FileURI">file:///home/user/test/in</parameter></div>
- <div class="xmlConf"><parameter name="transport.vfs.FileURI">sftp://bob:password@ftp.example.org/test/in</parameter></div>
+ <div class="xmlConf"><parameter name="transport.vfs.FileURI">sftp://bob:password@example.com/logs</parameter></div>
</td>
<td>Yes</td>
<td>N/A</td>
@@ -190,7 +200,7 @@
Specify the location to which the files should be moved after successfully
processing them. Required if transport.vfs.ActionAfterProcess is set to
MOVE. Ignored otherwise. Value must be a valid URI (local or remote).
- <div class="xmlConf"><parameter name="transport.vfs.MoveAfterProcess">file:///home/user/test/original</parameter></div>
+ <div class="xmlConf"><parameter name="transport.vfs.MoveAfterProcess">file:///home/test/original</parameter></div>
</td>
<td>No</td>
<td>N/A</td>
@@ -298,74 +308,79 @@
<td>false</td>
</tr>
<tr>
- <td>transport.vfs.UseTempFile</td>
- <td>If this parameter is set to true, the transport will create a temporary, then
- will transfer the content and it will be renamed to the configured file name. This
- is specially useful to avoid processing of partially transferred files using VFS
- transport. This configuration should be configured as part of the endpoint
- deceleration.</td>
- <td>No</td>
- <td>false</td>
- </tr>
- <tr>
<td>transport.vfs.MaxRetryCount</td>
- <td>The number of attempts that the polling task should attempt to poll the remote
- VFS destination after an error occurs when trying to resolve the remote file.
- For each erroneous attempts polling will be suspended for the amount given by
- the parameter 'transport.vfs.ReconnectTimeout'.
- <div class="xmlConf"><parameter name="transport.vfs.MaxRetryCount">3</parameter></div>
+ <td>
+ If the file transport listener encounters an error while trying to
+ read a file, it will try to read the file again after some time. This
+ parameter sets the maximum number of times the listener should retry
+ before giving up. Use the <a href="#transport.vfs.ReconnectTimeout">transport.vfs.ReconnectTimeout</a>
+ parameter to set the time duration between retries.
+ <div class="xmlConf"><parameter name="transport.vfs.MaxRetryCount">3</parameter></div>
</td>
<td>No</td>
<td>3</td>
</tr>
<tr>
- <td>transport.vfs.ReconnectTimeout</td>
- <td>The amount the current polling task should be suspended after the failed
- attempt to resolve the file object.
- <div class="xmlConf"><parameter name="transport.vfs.ReconnectTimeout">30000</parameter></div>
+ <td>transport.vfs.ReconnectTimeout <a name="transport.vfs.ReconnectTimeout"/></td>
+ <td>
+ The amount of time (in seconds) the current polling task should be
+ suspended for after a failed attempt to resolve a file.
+ <div class="xmlConf"><parameter name="transport.vfs.ReconnectTimeout">30000</parameter></div>
</td>
<td>No</td>
- <td>30000 milliseconds</td>
+ <td>30</td>
</tr>
<tr>
- <td>transport.vfs.FailedRecordsFileName</td>
- <td>Once the processing is done on a file it will be moved(to a new file) or
- deleted. If this operation fails, a log can be written with the cause to the
- given fail record file.
- <div class="xmlConf"><parameter name="transport.vfs.FailedRecordsFileName">vfs-move-failed-records.properties</parameter></div>
+ <td>transport.vfs.FailedRecordsFileName <a name="transport.vfs.FailedRecordsFileName"/></td>
+ <td>
+ Once a file has been fully processed, it will be moved to a new
+ location or deleted. If this operation fails, a log entry with the
+ failure details can be written to a separate log file. This parameter
+ controls the name of this failure log file.
+ <div class="xmlConf"><parameter name="transport.vfs.FailedRecordsFileName">move-errors.txt</parameter></div>
</td>
<td>No</td>
<td>vfs-move-failed-records.properties</td>
</tr>
<tr>
<td>transport.vfs.FailedRecordsFileDestination</td>
- <td>The destination to create the log file of failed move or delete operations
- on processed files.
- <div class="xmlConf"><parameter name="transport.vfs.FailedRecordsFileDestination">repository/conf</parameter></div>
+ <td>
+ Once a file has been fully processed, it will be moved to a new
+ location or deleted. If this operation fails, a log entry with the
+ failure details can be written to a separate log file. This parameter
+ controls the location (directory path) of this failure log file. To set
+ the name of the log file use the <a href="#transport.vfs.FailedRecordsFileName">transport.vfs.FailedRecordsFileName</a>
+ parameter.
+ <div class="xmlConf"><parameter name="transport.vfs.FailedRecordsFileDestination">logs/</parameter></div>
</td>
<td>No</td>
<td>repository/conf</td>
</tr>
<tr>
<td>transport.vfs.FailedRecordNextRetryDuration</td>
- <td>Once a move operation failed, the move task will be retried after this
- configured amount of time.
- <div class="xmlConf"><parameter name="transport.vfs.FailedRecordsFileName">vfs-move-failed-records.properties</parameter></div>
+ <td>
+ When a move operation has failed, the operation will be retried after this
+ amount of time (configured in milliseconds).
+ <div class="xmlConf"><parameter name="transport.vfs.FailedRecordNextRetryDuration">5000</parameter></div>
</td>
<td>No</td>
- <td>3000 milliseconds</td>
+ <td>3000</td>
</tr>
<tr>
<td>transport.vfs.MoveAfterFailedMove</td>
- <td>The destination to move the file after a failed move attempt.
+ <td>
+ The destination to move the file after a failed move attempt.
+ <div class="xmlConf"><parameter name="transport.vfs.MoveAfterFailedMove">repository/move-errors</parameter></div>
</td>
<td>No</td>
<td>N/A</td>
</tr>
<tr>
<td>transport.vfs.MoveFailedRecordTimestampFormat</td>
- <td>The time stamp format to report the failed move operation in the log.
- <div class="xmlConf"><parameter name="transport.vfs.MoveFailedRecordTimestampFormat">dd/MM/yyyy/ HH:mm:ss</parameter></div>
+ <td>
+ The time stamp format to use when reporting failed move operations in
+ the log.
+ <div class="xmlConf"><parameter name="transport.vfs.MoveFailedRecordTimestampFormat">HH:mm:ss</parameter></div>
</td>
<td>No</td>
<td>dd/MM/yyyy/ HH:mm:ss</td>
@@ -413,6 +428,8 @@
<tt>ftp://myusername:mypassword@somehost/pub/downloads/somefile.tgz</tt>
</li>
</ul>
+
+ <h4>File Locking <a name="Locking"/></h4>
<p>
By default file locking is globally enabled for the file transport sender.
This behavior can be overridden at the endpoint level by specifying
@@ -426,6 +443,8 @@
You may also change the global locking behavior by setting the <tt>transport.vfs.Locking</tt>
parameter in the file transport sender configuration in axis2.xml file.
</p>
+
+ <h4>FTP Passive Mode <a name="Passive"/></h4>
<p>
When writing to remote file locations using a protocol such as FTP, you might
want Synapse to communicate with the FTP server in the passive mode. To
@@ -435,6 +454,70 @@
<div class="xmlConf"><endpoint>
<address uri="vfs:ftp://myusername:mypassword@somehost/pub/downloads/somefile.tgz?vfs.passive=true"/>
</endpoint></div>
+
+ <h4>Retrying on Error <a name="Retry"/></h4>
+ <p>
+ When the file transport sender encounters an error while trying to write a file,
+ it can retry after some time. This is useful to recover from certain types of
+ transient I/O errors and network connectivity issues. Following parameters
+ can be configured as URL query parameters on the file (vfs) endpoints to
+ make use of this feature.
+ </p>
+ <table>
+ <tr>
+ <th>Parameter Name</th>
+ <th>Description/Example</th>
+ <th>Required</th>
+ <th>Default</th>
+ </tr>
+ <tr>
+ <td>transport.vfs.MaxRetryCount</td>
+ <td>
+ Maximum number of retries to perform before giving up.
+ </td>
+ <td>No</td>
+ <td>3</td>
+ </tr>
+ <tr>
+ <td>transport.vfs.ReconnectTimeout</td>
+ <td>
+ Time duration (in seconds) between retry attempts.
+ </td>
+ <td>No</td>
+ <td>30</td>
+ </tr>
+ </table>
+
+ <h4>Using Temporary Files <a name="transport.vfs.UseTempFile"/></h4>
+ <p>
+ The file transport sender does not write file content atomically. Therefore a
+ process reading a file updated by Synapse, may read partial
+ content. To get around this limitation, the temporary file support can be
+ activated on the target file (vfs) endpoint:
+ </p>
+ <div class="xmlConf"><endpoint>
+ <address uri="vfs:file:///var/spool/synapse/out?transport.vfs.UseTempFile=true"/>
+</endpoint></div>
+ <p>
+ This forces the file transport sender to write the data to a temporary file and
+ then move the temporary file to the actual destination configured in the file
+ endpoint. On most operating systems (e.g. Unix/Linux, Windows), this delivers the
+ desired atomic file update behavior. When the file endpoint points to a remote
+ file system, the temporary files will be created on the remote file system, thus
+ preserving the atomic update behavior.
+ </p>
+
+ <h4>Appending to Files <a name="Append"/></h4>
+ <p>
+ When updating an existing file, the file transport sender usually overwrites the
+ old content. To get append behavior instead, set transport.vfs.Append parameter
+ on the target endpoint:
+ </p>
+ <div class="xmlConf"><endpoint>
+ <address uri="vfs:file:///var/spool/synapse/out?transport.vfs.Append=true"/>
+</endpoint></div>
+
+ <h4>Out-only Message Exchange Pattern <a name="OutOnly"/></h4>
<p>
It should be noted that by its nature, the file transport sender doesn't support
synchronous responses and should only be invoked using the out-only message
@@ -489,6 +572,8 @@
</p>
<p>
It should also be noted that the VFS transport sender doesn't create files atomically.
+ Use the <a href="#transport.vfs.UseTempFile">transport.vfs.UseTempFile</a> endpoint
+ parameter to get around this issue.
</p>
</section>
</body>