Zaku Task Queue｣

Zaku is a light-weight task queue backed by redis for machine learning workloads.

To Install:

pip install 'zaku[all]==0.0.15-rc3'

Here is an example of how to add and retrieve jobs from Zaku. For a more comprehensive list of examples, please refer to the examples page.

from zaku import TaskQ

app = TaskQ(uri="http://localhost:9000")

task_id, task = app.take()
try:
  print("do your things", task)
except Except as e:
  app.mark_reset(task_id)
  raise e
else:
  app.mark_done(task_id)

Life Cycle of a task

A task after being created and inserted into the queue, will be taken by a worker, processed, and then marked as done or reset. If the task is marked as done, it will be removed from the queue. If the task is marked as reset, it will be put back into the queue.

In the event of you forgetting to mark the task as done or reset, the task will be marked as reset after a certain amount of time. This is to prevent the task from being stuck in the queue forever. This time is called ttl or time-to-live.

Zaku is built by researchers at MIT in fields including robotics, computer vision, and computer graphics.

  • light-weight and performant

  • scalable and versatile.

  • Open source, licensed under MIT

To get a quick overview of what you can do with zaku, check out the following:

  1. take a look at the basic tutorial or the tutorial for robotics:

  1. or try to take a look at the example gallery here

For a comprehensive documentation on the API, please refer to the API documentation on Components | zaku.