Long may you run

By Serdar Yegulalp | 2017/09/03 14:07

One thing I've been wrestling with on and off is how to deal with long-running operations that are potentially blocking. For instance, deleting tags. If you have to delete a tag and re-queue 200 pages because of the deletion, the re-queuing operation can be quite expensive, and it can hang input for a long time.

The current solution I have for this is to allow such operations to be operator-batched. That is, we do the queuing and deletion in batches of, say, 25 or 50, with either the operator re-confirming at the end of each batch or using an AJAX page to keep things humming. It's not ideal, because it brings up all kinds of questions of how to lock things that are not necessarily directly affected by these operations (what if someone adds a tag to a page while it's being deleted?). But I'd rather have some plan for how to do this properly in the long run than no plan at all.

Tags: queue

comments powered by Disqus