I'm coming at it from the Ruby angle, in which jobs are often triggered using ActiveRecord after_commit hooks. I admit to being ignorant of the Python/Celery way of doing things so perhaps I am missing the point. I'm talking about jobs being produced atomically with the data that necessitated the background job (I realize not all background jobs are spawned in this fashion).
I agree with your point about polling being bad, however as someone pointed out below it's not an issue with Postgres's LISTEN/NOTIFY (and I added a note to the queue_classic gem which makes this easy to take advantage of in MRI Ruby).
Obviously I'm aware that Redis and RabbitMQ persist jobs. That's not what I was talking about at all.
I think we're on different wavelengths here so I'll let it be. :-)
I'm not aware of how ruby does it other then hearing about 2 most popular solutions that are used with rails.
As for Postgres's PUB/SUB I've replied to the reply that mentioned that and you can't really use that. Would be great if we cold update the broker driver and see if we could get support though.
I agree with your point about polling being bad, however as someone pointed out below it's not an issue with Postgres's LISTEN/NOTIFY (and I added a note to the queue_classic gem which makes this easy to take advantage of in MRI Ruby).
Obviously I'm aware that Redis and RabbitMQ persist jobs. That's not what I was talking about at all.
I think we're on different wavelengths here so I'll let it be. :-)