Coalescing NSOperations

This post is now out dated, please instead checkout http://williamboles.me/removing-bolierplate-when-coalescing-nsoperations/ for my latest solution to coalescing operations - I'm leaving it here just incase anyone needs it.


In a previous post I wrote about combing NSURLSession with NSOperation to wrap the requesting and responding to networking requests into one single task. One of the great things about this approach is that it allowed you more control over your networkings such as canceling them easily when the user logs out. Another advantage of this approach which I will explore below is how to coalesce these NSOperation subclasses so as to ensure that you squeeze as much from the bandwidth available.

(This approach doesn't just work for networking NSOperation subclasses but rather than any NSOperation subclass - in fact the example below doesn't contain any networking code)

Brief recap 💪

Let's have a brief recap on the architecture we explored in the previous post:

Class diagram

Making introductions

'To coalesce' is defined by the Oxford Dictionary as to:

combine (elements) in a mass or whole

When we coalesce our NSOperation subclasses our objective is to only execute the task that the NSOperation encompasses once within the time it takes for that operation to get to the front of the queue and be executed. We want to also ensure that only those classes that need to know about coalescing, actually do know about it. To do this we need to ensure that if a object asks for the same operation to be queued over and over again then it receives the same number of callbacks as it made requests. Let's begin to explore how we can do that by looking briefly at the classes we will later explore in more depth:

  • QueueManager
    • Responsible for queuing operations, storing all closures that will be executed and determining if an operation already exists on the queue.
  • CoalescingExampleManager
    • Responsible for scheduling operations, adding closure to QueueManager and executing all (relevant) closures once the operation has completed. Ensures that only unique operations are created.
  • CoalescingOperation
    • Parent class that stores the identifier used to coalesce closures/operation
  • CoalescingExampleOperation
    • An example of a CoalescingOperation subclass to show how we can use this coalescing technique.
Getting to know each other more

Now that the introductions have been made, let's explore what we have:

class CoalescingExampleManager: NSObject {

    // MARK: - Add

    class func addExampleCoalescingOperation(queueManager: QueueManager = QueueManager.sharedInstance, completion: (QueueManager.CompletionClosure)?) {
        let coalescingOperationExampleIdentifier = "coalescingOperationExampleIdentifier"

        if let completion = completion {
            queueManager.addNewCompletionClosure(completion, identifier: coalescingOperationExampleIdentifier)
        }

        if !queueManager.operationIdentifierExistsOnQueue(coalescingOperationExampleIdentifier) {
            let operation = CoalscingExampleOperation()
            operation.identifier = coalescingOperationExampleIdentifier
            operation.completion = {(successful) in
                let closures = queueManager.completionClosures(coalescingOperationExampleIdentifier)

                if let closures = closures {
                    for closure in closures {
                        closure(successful: successful)
                    }

                    queueManager.clearClosures(coalescingOperationExampleIdentifier)
                }
            }

            queueManager.enqueue(operation)
        }
    }
}

Please note that I've chosen very generic names in this example however this case would more typically be called something like e.g. FeedAPIManager or UserManager.

The above manager will schedule the operation to be executed by passing it to the QueueManager. There is a lot happening in the above class so let's work through the unique parts.

class func addExampleCoalescingOperation(queueManager: QueueManager = QueueManager.sharedInstance, completion: (QueueManager.CompletionClosure)?)

Above is the method's signature which accepts two parameters: queueManager and completion. queueManager is the object that the soon to be created operation will be in enqueued on. completion is a closure that will be called when the operation has finished executing and it's also the way we will coalesce multiple calls together.

let coalescingOperationExampleIdentifier = "coalescingOperationExampleIdentifier"

Each operation needs to have it's own unique identifier so that we can determine if the operation is already in the queue.

if let completion = completion {
    queueManager.addNewCompletionClosure(completion, identifier: coalescingOperationExampleIdentifier)
}

Our coalescing approach is built on taking the closure (or closures) from an operation storing them outside of the operation, injecting our own closure into operation and then iterating through the original closures and triggering each one individually to ensure that all interested parties are informed as the outcome of the operation. The above code snippet is our first step in this process by taking the original closure and passing it to the QueueManager which will store it alongside the operation identifier.

if !queueManager.operationIdentifierExistsOnQueue(coalescingOperationExampleIdentifier) {

Next we check if the operation is already on our queue. If the operation is already on the queue all job here is done else we still have some more work to do.

let operation = CoalscingExampleOperation()
operation.identifier = coalescingOperationExampleIdentifier

Here we create the operation itself and assign it the same identifier value that we used for coalescing the completion block.

operation.completion = {(successful) in
    let closures = queueManager.completionClosures(coalescingOperationExampleIdentifier)

    if let closures = closures {
        for closure in closures {
            closure(successful: successful)
        }

        queueManager.clearClosures(coalescingOperationExampleIdentifier)
    }
}

Ok, so here we inject our own completion closure into the operation which takes all the coalesced closures and executes them one by one. When all are executed, it then clears the closures from our queue manager so ensuring that we don't accidentally coalesce closures from an already executed operation with a new operation's closures.

So in the above class we use QueueManager a great deal for such a small method so let's explore this one. For my more sophisticated and charming readers who have read the previous post I linked to at the start of this post QueueManager is a class that are already familiar with (if you want to be sophisticated and charming just go back and give a read through 😏).

class QueueManager: NSObject {

    typealias CompletionClosure = (successful: Bool) -> Void

    // MARK: - Accessors

    lazy var queue: NSOperationQueue = {
        let queue = NSOperationQueue()

        return queue;
    }()

    lazy var completionClosures: [String: [CompletionClosure]] = {
        let completionClosures = [String: [CompletionClosure]]()

        return completionClosures
    }()

    // MARK: - SharedInstance

    static let sharedInstance = QueueManager()

    // MARK: Addition

    func enqueue(operation: NSOperation) {
        queue.addOperation(operation)
    }

    // MARK: - Callbacks

    func addNewCompletionClosure(completion: (CompletionClosure), identifier: String) {
        var closures = completionClosures[identifier] ?? [CompletionClosure]()

        closures!.append(completion)
        completionClosures[identifier] = closures!
    }

    func completionClosures(identifier: String) -> [CompletionClosure]? {
        return completionClosures[identifier]
    }

    // MARK: Existing

    func operationIdentifierExistsOnQueue(identifier: String) -> Bool {
        let operations = self.queue.operations

        let identifiers = (operations as! [CoalscingOperation]).map{$0.identifier}
        let exists = identifiers.contains({ identifier == $0 })

        return exists
    }

    // MARK: Clear

    func clearClosures(identifier: String) {
        completionClosures.removeValueForKey(identifier)
    }
}

Ok, I understand you're a busy person and don't have the time to go back through the previous post (especially as it's in objective-c 😱) so I'll take us through it as well. The QueueManager class is a singleton that holds all of our queues and stores the closures that we will coalesce. So let's look through the more unique parts:

typealias CompletionClosure = (successful: Bool) -> Void

The above is a generic closure declaration that we will give us greater control over what closures this class will accept.

lazy var completionClosures: [String: [CompletionClosure]] = {
    let completionClosures = [String: [CompletionClosure]]()

    return completionClosures
}()

A lazy loaded dictionary that will store the closures waiting to be executed - with the operation identifier as the key and the actual closures, in an array, as value.

func addNewCompletionClosure(completion: (CompletionClosure), identifier: String) {
    var closures = completionClosures[identifier] ?? [CompletionClosure]()

    closures!.append(completion)
    completionClosures[identifier] = closures!
}

Nothing overly special about the above, we ensure that when adding a closure we create an array to place the closure in, which is then placed into the overall dictionary.

func completionClosures(identifier: String) -> [CompletionClosure]? {
    return completionClosures[identifier]
}

The above returns a subset of the closures stored that match the identifier.

func operationIdentifierExistsOnQueue(identifier: String) -> Bool {
    let operations = self.queue.operations

    let identifiers = (operations as! [CoalscingOperation]).map{$0.identifier}
    let exists = identifiers.contains({ identifier == $0 })

    return exists
}

Here we inspect the operation currently on the queue and determine if an operation already exists the passed in identifier.

func clearClosures(identifier: String) {
    completionClosures.removeValueForKey(identifier)
}

A tidying up method to clear away used closures.

Now for the final piece in the jigsaw, the changes necessary to the operation to support.

class CoalescingExampleOperation: NSOperation {

    // MARK: Accessors

    var identifier: String?
}

A simple string property.

Bringing it to an end

With this approach we can ensure that we only execute unique operation within the context of the queue while still supporting parallel processing overall. In the above example, we only support one queue and one type of closure but the QueueManager can easily be extended by adding more properties with the same pattern as shown.

Nothing however comes free of charge. We can see in the above that scheduling an operation requires more setup code than a non-coalesced operation and that storing all of those closures will increase the memory consumption of your app.

You can find the completed project by heading over to https://github.com/wibosco/CoalescingOperations-Example