You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "James A. Robinson" <ji...@stanford.edu> on 2012/08/08 18:03:56 UTC
removing broker files (other than using the retention policy)?
Hi folks,
I'm hunting around on the kafka site and haven't managed to figure out
whether or not there are tools to allow one to safely delete the
/tmp/kafka-logs files. Are there tools outside the retention policy?
I'm running tests on a somewhat limited machine, and I'd like to
figure out a way to cleanly zero everything out after a test run. It
looks as though kafka is keeping a filehandle open for all the logs,
and it looks as though it's keeping track of offsets in zookeeper, so
I figure it isn't safe to simply delete the files!
Jim
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
James A. Robinson jim.robinson@stanford.edu
Stanford University HighWire Press http://highwire.stanford.edu/
+1 650 7237294 (Work) +1 650 7259335 (Fax)
Re: removing broker files (other than using the retention policy)?
Posted by Casey Sybrandy <Ca...@six3systems.com>.
Hello,
I've found that stopping kafka, deleting everything out of the
kafka-logs directory, then restarting kafka works fine. I've done this
on a development VM without any issues. However, if there's a
better/easier way, I'd love to know about it as well.
Casey
On 8/8/2012 12:03 PM, James A. Robinson wrote:
> Hi folks,
>
> I'm hunting around on the kafka site and haven't managed to figure out
> whether or not there are tools to allow one to safely delete the
> /tmp/kafka-logs files. Are there tools outside the retention policy?
>
> I'm running tests on a somewhat limited machine, and I'd like to
> figure out a way to cleanly zero everything out after a test run. It
> looks as though kafka is keeping a filehandle open for all the logs,
> and it looks as though it's keeping track of offsets in zookeeper, so
> I figure it isn't safe to simply delete the files!
>
> Jim
>
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> James A. Robinson jim.robinson@stanford.edu
> Stanford University HighWire Press http://highwire.stanford.edu/
> +1 650 7237294 (Work) +1 650 7259335 (Fax)
Re: removing broker files (other than using the retention policy)?
Posted by "James A. Robinson" <ji...@stanford.edu>.
On Wed, Aug 8, 2012 at 10:25 AM, Neha Narkhede <ne...@gmail.com> wrote:
> If you want to clean the log files after a test run, assuming that
> no producers/consumers are running, you can simply delete the log
> directory
Ok, that's what I'll do then. Thank you!
Jim
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
James A. Robinson jim.robinson@stanford.edu
Stanford University HighWire Press http://highwire.stanford.edu/
+1 650 7237294 (Work) +1 650 7259335 (Fax)
Re: removing broker files (other than using the retention policy)?
Posted by Neha Narkhede <ne...@gmail.com>.
Jim,
Currently, we don't have a tool to forcefully delete log files. If you
want to clean the log files after a test run, assuming that no
producers/consumers are running, you can simply delete the log
directory. The next time a kafka server starts up, it will recreate
the log directory. If you want to delete some log files while a Kafka
server is running, assuming that there are no consumers running, you
can delete all but the latest log segment for each partition and then
restart the Kafka server. If you have consumers actively consuming
data from older log segments, they will run into errors if you delete
the log segments that the server was trying to serve messages from.
Thanks,
Neha
On Wed, Aug 8, 2012 at 9:03 AM, James A. Robinson
<ji...@stanford.edu> wrote:
> Hi folks,
>
> I'm hunting around on the kafka site and haven't managed to figure out
> whether or not there are tools to allow one to safely delete the
> /tmp/kafka-logs files. Are there tools outside the retention policy?
>
> I'm running tests on a somewhat limited machine, and I'd like to
> figure out a way to cleanly zero everything out after a test run. It
> looks as though kafka is keeping a filehandle open for all the logs,
> and it looks as though it's keeping track of offsets in zookeeper, so
> I figure it isn't safe to simply delete the files!
>
> Jim
>
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> James A. Robinson jim.robinson@stanford.edu
> Stanford University HighWire Press http://highwire.stanford.edu/
> +1 650 7237294 (Work) +1 650 7259335 (Fax)
Re: removing broker files (other than using the retention policy)?
Posted by Joel Koshy <jj...@gmail.com>.
Hi Jim,
We currently don't have a tool for that in 0.7/trunk. So you will have to
bring the broker down, delete the topic and bring the broker back up. 0.8
provides an admin tool to delete topics.
Joel
On Wed, Aug 8, 2012 at 9:03 AM, James A. Robinson <jim.robinson@stanford.edu
> wrote:
> Hi folks,
>
> I'm hunting around on the kafka site and haven't managed to figure out
> whether or not there are tools to allow one to safely delete the
> /tmp/kafka-logs files. Are there tools outside the retention policy?
>
> I'm running tests on a somewhat limited machine, and I'd like to
> figure out a way to cleanly zero everything out after a test run. It
> looks as though kafka is keeping a filehandle open for all the logs,
> and it looks as though it's keeping track of offsets in zookeeper, so
> I figure it isn't safe to simply delete the files!
>
> Jim
>
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> James A. Robinson jim.robinson@stanford.edu
> Stanford University HighWire Press http://highwire.stanford.edu/
> +1 650 7237294 (Work) +1 650 7259335 (Fax)
>