Can I use lpop/rpop to create a simple queue system with Redis? -

i tried several message/job queue systems seem add unnecessary complexity , end queue process dying no reason , cryptic log messages.

so want make own queue system using redis. how go doing this?

from have read, redis because has lpop , rpush methods, , pub/sub system used notify workers there new messages consumed. correct?

yes can. in fact there number of package ... including celery , rq python , resque ruby , ports of resque java (jesque , javascript (coffee-resque).

there's restmq implemented in python, designed use restful system.

there many others.

note redis lists simplest possible network queuing system. however, making things robust on simple primitives offered redis non-trivial (and may impossible values of "robust" --- @ least on server side). many of these libraries using redis queue add features , protocols intended minimize chances of lost messages while ensuring "at-most-once" semantics. many of these use rpoplpush redis primitive other processing on secondary list handle acknowledgement of completed work , re-dispatch of "lost" units. (consider case client "popped" work unit off queue , died before work results posted; how detect , mitigate scenario?)

in cases people have cooked elaborate bits of server side (redis lua eval) scripting handle more reliable queuing. example implementing rpoplpush replacing "push" zadd (thus adding item , timestamp "sorted set" representing work that's "in progress"). in such systems work completed zrem , scanned "lost" work using zrangebyscore.

here thoughts on topic of implementing robust queuing system salvatore sanfilippo (a.k.a. antirez, author of redis): adventures in message queues discusses considerations , forces led him work on disque.

i'm sure you'll find detractors argue redis poor substitute "real" message bus , queuing system (such rabbitmq). salvatore says in 'blog entry, , i'd welcome others here spell out cogent reasons preferring such systems.

my advice start redis during prototyping; keep use of system abstracted consolidated bit of code. celery, among others, you. can start using celery redis backend , readily replace backend rabbitmq or others little effect on bulk of code.

for catalog of alternatives, consider perusing: http://queues.io/