[jira] [Commented] (FLUME-2613) Tool/script for deleting individual message from queue

["Ashish Paliwal (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14322833#comment-14322833 ] 

Ashish Paliwal commented on FLUME-2613:
---------------------------------------

I couldn't find a way to easily do the job using existing code in FileChannel Integrity tool. Most of the file that could be used have package scope in file channel. Found one way to do this

1. Reuse existing File Channel integrity tool code base
2. Inside the loop where file is verified for non-corrupted events, open a RandomAccessFile on the same datadir in read only mode
3. We get event position inside the loop (already present), we reuse the pointer to seek to the offset from Step#2
4. Reuse LogFile.java#get(int offset) code to get the Flume Event
5. Apply the user supplied logic to validate Event data
6. If Event is invalid, we mark record as NOOP

[~hshreedharan]/[~roshan_naik] Any suggestions on the approach? I am going nuts trying for figure an easy solution for this. 

> Tool/script for deleting individual message from queue
> ------------------------------------------------------
>
>                 Key: FLUME-2613
>                 URL: https://issues.apache.org/jira/browse/FLUME-2613
>             Project: Flume
>          Issue Type: Improvement
>            Reporter: Charles McLaughlin
>            Assignee: Ashish Paliwal
>            Priority: Minor
>
> We had a situation where one of our Flume agents got stuck on a message due to unexpected format.  To get things moving again, I stopped the Flume agent, moved the file backed channel data out of the way and re-started the Flume agent.  I'd like to pop the bad message from the queue data on disk and ideally there would be a recommended tool/script.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)