You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "zhouqingfa (JIRA)" <ji...@apache.org> on 2012/12/03 04:03:58 UTC
[jira] [Created] (FLUME-1764) File_roll type Sink how to achieve
according to multiple files dynamically generated directory, and the file
on the corresponding directory
zhouqingfa created FLUME-1764:
---------------------------------
Summary: File_roll type Sink how to achieve according to multiple files dynamically generated directory, and the file on the corresponding directory
Key: FLUME-1764
URL: https://issues.apache.org/jira/browse/FLUME-1764
Project: Flume
Issue Type: Improvement
Components: Sinks+Sources
Affects Versions: v1.4.0
Reporter: zhouqingfa
Fix For: v1.4.0
Through the execSource in the tail -f command to tail multiple files ( via a script file, the script file using the " * " wildcard ), tail files can be varied according to the file name into a dynamically generated directory under the file name directory name ( according to the file name is named, not must with the file name name, for example: file name is access_1024.log, and directory name is 1024 ), Sink use file_roll types, such as function can be realized? Thanks......
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1764) File_roll type Sink how to achieve
according to multiple files dynamically generated directory, and the file
on the corresponding directory
Posted by "Roshan Naik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509058#comment-13509058 ]
Roshan Naik commented on FLUME-1764:
------------------------------------
The description in this jira is not clear. But appears to have something in common with [FLUME-1161|https://issues.apache.org/jira/browse/FLUME-1661]
> File_roll type Sink how to achieve according to multiple files dynamically generated directory, and the file on the corresponding directory
> -------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLUME-1764
> URL: https://issues.apache.org/jira/browse/FLUME-1764
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.4.0
> Reporter: zhouqingfa
> Fix For: v1.4.0
>
>
> Through the execSource in the tail -f command to tail multiple files ( via a script file, the script file using the " * " wildcard ), tail files can be varied according to the file name into a dynamically generated directory under the file name directory name ( according to the file name is named, not must with the file name name, for example: file name is access_1024.log, and directory name is 1024 ), Sink use file_roll types, such as function can be realized? Thanks......
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1764) File_roll type Sink how to achieve
according to multiple files dynamically generated directory, and the file
on the corresponding directory
Posted by "zhouqingfa (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhouqingfa updated FLUME-1764:
------------------------------
Description: I want to achieve such a function：Source terminal(using the 'tail -f' command of execSource)can dynamicly monitor a directory to get new files(such as access_1045.log),then transfer immediately the new file to the local directory（using FileRollSink type） and the file will reache such a directory（The directory will be based on the file name dynamically generated,for example,in the configuration file,sink.directory=/opt/apps_install,then the file will be put such a directory /opt/apps_install/1045）.Finally,Source terminal can not only minotor a directory and transter the new file to /opt/apps_install/......，but also transfer immediately the new content of the file to the Corresponding file in the directory.How to achieve such a function in the flume1.4.0. Thanks...... (was: Through the execSource in the tail -f command to tail multiple files ( via a script file, the script file using the " * " wildcard ), tail files can be varied according to the file name into a dynamically generated directory under the file name directory name ( according to the file name is named, not must with the file name name, for example: file name is access_1024.log, and directory name is 1024 ), Sink use file_roll types, such as function can be realized? Thanks......)
> File_roll type Sink how to achieve according to multiple files dynamically generated directory, and the file on the corresponding directory
> -------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLUME-1764
> URL: https://issues.apache.org/jira/browse/FLUME-1764
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.4.0
> Reporter: zhouqingfa
> Fix For: v1.4.0
>
>
> I want to achieve such a function：Source terminal(using the 'tail -f' command of execSource)can dynamicly monitor a directory to get new files(such as access_1045.log),then transfer immediately the new file to the local directory（using FileRollSink type） and the file will reache such a directory（The directory will be based on the file name dynamically generated,for example,in the configuration file,sink.directory=/opt/apps_install,then the file will be put such a directory /opt/apps_install/1045）.Finally,Source terminal can not only minotor a directory and transter the new file to /opt/apps_install/......，but also transfer immediately the new content of the file to the Corresponding file in the directory.How to achieve such a function in the flume1.4.0. Thanks......
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1764) File_roll type Sink how to achieve
according to multiple files dynamically generated directory, and the file
on the corresponding directory
Posted by "Brock Noland (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509820#comment-13509820 ]
Brock Noland commented on FLUME-1764:
-------------------------------------
Echo'ing Roshan, could you explain again exactly what the problem is?
> File_roll type Sink how to achieve according to multiple files dynamically generated directory, and the file on the corresponding directory
> -------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLUME-1764
> URL: https://issues.apache.org/jira/browse/FLUME-1764
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.4.0
> Reporter: zhouqingfa
> Fix For: v1.4.0
>
>
> I want to achieve such a function：Source terminal(using the 'tail -f' command of execSource)can dynamicly monitor a directory to get new files(such as access_1045.log),then transfer immediately the new file to the local directory（using FileRollSink type） and the file will reache such a directory（The directory will be based on the file name dynamically generated,for example,in the configuration file,sink.directory=/opt/apps_install,then the file will be put such a directory /opt/apps_install/1045）.Finally,Source terminal can not only minotor a directory and transter the new file to /opt/apps_install/......，but also transfer immediately the new content of the file to the Corresponding file in the directory.How to achieve such a function in the flume1.4.0. Thanks......
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira