You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Vivek Goel <go...@gmail.com> on 2012/09/18 13:02:43 UTC
How to schedule a event in future time
Hi,
My problem description is follows:
I have n state based database *infinite* crawlers:
Currently how it is happening:
- We are using single machine for crawling.
- We have three level of priority queue. High, Medium and LOW.
- At starting all Database job are put into lower level queue.
- Worker reads a job from queue and do operation.
- After finishing job it reschedule it with a delay of 5 minutes.
Solution I found
For Priority Queue I can use:
-
http://zookeeper.apache.org/doc/r3.2.2/recipes.html#sc_recipes_priorityQueues
Problem solution I am still searching are:
1. How to reschedule a job in queue with future schedule time. Is there
a way to do that in zookeeper ?
2. Canceling a already started job. Suppose user change his database
authentication details. I want to stop already running job for that
database and restart with new details.
What I thought is while starting a worker It will subscribe for that
it's znode changes and if something happen, It will stop that job and
reschedule it.
3. Infinite Queue
What I thought is that after finishing it will remove it from queue and
readd it with future schdule time. (It implementation depend on point 1)
Is it correct way of doing this task infinite task?
regards
Vivek Goel