python - Use celery priority queue with broadcast tasks -
i want implement tasks priority in celery workers. can create different queues high priority tasks , low priority tasks. need send broadcast tasks workers broadcast queue , not working. here tasks.py file:
from celery import celery kombu.common import broadcast, queue, exchange app = celery('tasks') app.conf.update( celery_result_backend='amqp', celery_accept_content=['json'], celery_task_serializer='json', celery_result_serializer='json', broker_url='amqp://', celery_queues=(queue('default', exchange('default'), routing_key='default'), queue('low_priority', exchange('low_priority'), routing_key='low_priority'), broadcast('broadcast_tasks'), ), celery_routes={'tasks.broadcast': {'queue': 'broadcast_tasks'}, 'tasks.low_task': {'queue': 'low_priority'}, }, celery_default_queue = 'default', celery_default_exchange = 'default', celery_default_routing_key = 'default' ) @app.task def broadcast(): print "broadcast called" @app.task def low_task(): print "low priority called" @app.task def def_task(): print "default called"
when run celery workers command:
celery -a tasks -q default worker --loglevel=info celery -a tasks -q default,low_priority worker --loglevel=info
tasks priority works broadcast tasks not acknowledged.
when run command without queue argument broadcast working tasks priority not:
celery -a tasks worker --loglevel=info celery -a tasks worker --loglevel=info
as understand happens because broadcast queue have unique name, bcast.0b5dbce0-9bcb-48a5-8554-cbb7f32a6703
each worker.
does have workaround? in advance!
Comments
Post a Comment