Keeping things going when the user leaves

In the past iOS was always about living in the moment - totally focused on whatever the user was attempting to do. This approach was good news for users as it allowed for a very responsive user experience on performance constrained devices however as app developers it limited what we could do because as soon as an app went into the background iOS suspended it. As devices have become more powerful and energy-efficient, iOS has eased the "living in the moment" restrictions while maintaining responsiveness.

One way that iOS has eased restrictions is allowing apps to start/continue download and upload requests in the background - collectively known as background-transfers. Support for background-transfers was first introduced in iOS 7, so it's not a recent change, but it's such a powerful tool that I wanted to explore it further and especially show a solution for recovering from a terminated state.

Different types of sessions

When Apple introduced the URLSession suite of classes, it allowed the creation of multiple network sessions, each with their own configurations. This allowed for easily differentiating between the various types of network requests that an app has to make. The main session configurations are:

  1. Default - allowing for all types of network requests in the foreground.
  2. Ephemeral - similar to default but more forgetful (doesn’t write caches, cookies, or credentials to disk).
  3. Background - allowing for uploading and downloading content even when the app isn't running.

Of course, it's possible to further differentiate sessions within those configurations by adjusting the properties on each session.

At the time URLSession was introduced I was building a very media-heavy app, and the promise of being able to continue to upload or download content when the app wasn't running by using a background session was very appealing.

Peeking under the hood 🕵️

When scheduling a background-transfer on a background session, that transfer is passed to the nsurlsessiond daemon which runs as a separate process from the app to actually be actioned. As this daemon lives outside of the lifecycle of the app if the app is suspended or killed the transfer will continue unaffected - this is the main difference from scheduling an URLSessionDownloadTask or URLSessionUploadTask request on a default or ephemeral session where the request is tied to the lifecycle of the app. Once a background-transfer is complete, if the app has been suspended/terminated iOS will wake it up (in the background) and allow the app to perform any post-transfer work (within a limited time frame). If the app is in the foreground, control will be passed back to the app as if the transfer has been scheduled on default or ephemeral session (without a limited time frame for post-transfer processing).

Important to note here that you can schedule URLSessionDataTask requests on a background session but they will only be acted on when the app is in the foreground.

You're probably thinking:

"That sounds pretty amazing! Why isn't this the default behaviour for transfers?"

Well, there are a few reasons:

  • The more background processing that the device has to support the quicker the battery will drain and the more bandwidth that will be consumed. So while Apple understands that background processing is needed, it wants to ensure that as app developers we use it responsibly and only where it's actually adding value.

  • From a developer-POV supporting background-transfers requires a bit more programming than a foreground-only session. Rather than the simple closure based API that foreground-only sessions support, to use background sessions the developer needs to conform to various protocols. This additional programming while not hugely taxing (as we shall see shortly) does require each developer to know more about how URLSession works before they can make a network request. So in true Apple fashion, they have opted to not only keep it simple for the users but also for us developers.

Now that the hood is well and truly peeked under, let's get back to how to use background-transfers to provide a better experience for our users.

Scenarios

Any background-transfer solution should be able to handle 3 different transfer scenarios:

  1. Foreground - when the app is open, the transfer should behave the same way as if the session was using the default configuration.
  2. Suspended/Backgrounded - when the app is in a suspended state and the asset is downloaded, the app should be able to be woken up to finish the transfer.
  3. Terminated - when the app has been terminated by iOS, the app should be able to be relaunched to finish that transfer.

In the rest of this post, we will build an image download system that will look to handle the 3 scenarios described above. This example will populate a collectionview using data retrieved from Imgur with all image retrieval happening on a background session.

This post will focus solely on downloading using a background session as it's much easier to find services that allow us to retrieve content from their system than services that allow us to upload content.

This post will gradually build up to a working example however if you're on a tight deadline and/or there is murderous look creeping into your manager's eyes 😡, then head on over to the completed example and take a look at BackgroundDownloader, BackgroundDownloaderContext, BackgroundDownloadItem and AppDelegate to see how things end up - in order to run this you will need to follow the instructions below.

Let's get downloading

To support background-transfers we need to grant permission for Background Modes. This is done by opening the Capabilities tab in the target and switching the Background Modes toggle to ON.

As multiple viewcontrollers within the project could be retrieving images I decided to handle these download requests within their own class - BackgroundDownloader. BackgroundDownloader will be responsible for configuring our background session, scheduling download requests against that session and responding to any delegate callbacks.

Lets build the skeleton of the BackgroundDownloader class:

class BackgroundDownloader: NSObject {
    
    private var session: URLSession!

    // MARK: - Singleton

    static let shared = BackgroundDownloader()

    // MARK: - Init

    private override init() {
        super.init()

        let configuration = URLSessionConfiguration.background(withIdentifier: "background.download.session")
        session = URLSession(configuration: configuration, delegate: self, delegateQueue: nil)
    }
}

In the above code snippet, we create an instance of an URLSessionConfiguration instance and URLSession that will handle any background tasks. configuration is initialised using the background convenience init'er to allow any URLSession instances to support background-transfers. Each URLSessionConfiguration instances needs to have an identifier. This identifier plays a critical role in allowing a background-transfer that started on one instance of URLSession to be completed on another. This happens when the app has been terminated and iOS wakes the terminated app up to finish the transfer - configuration uses a static string as an identifier so it can be recreated (we will see this in action later). It's important to note here that each session needs to have a unique identifier within that app's execution lifecycle - if multiple sessions have the same identifier, iOS has the rather alarming warning in its documentation:

"You must create exactly one session per identifier (specified when you create the configuration object). The behaviour of multiple sessions sharing the same identifier is undefined."

"undefined" doesn't sound like a good state for the app to get into 😬 so let's avoid it.

As BackgroundDownloader is a singleton, only one session will have this identifier and all background-transfers will be scheduled on a session using that identifier.

An interesting difference between background sessions and the other session types is that with background sessions we can't use the completionHandler methods when creating a URLSessionDownloadTask task (doing so won't result in the compiler throwing an error, but the app will throw an exception at run time). Instead of using the closure approach, BackgroundDownloader will need to implement the URLSessionDownloadDelegate protocol (and eventually URLSessionDelegate) - this need to implement these delegates is the reason that BackgroundDownloader is a subclass of NSObject.

The more eagle-eyed among you will have noticed that the session property is implicitly unwrapped. Normally I try and avoid implicitly unwrapped properties but as session is a private property and is set in the init'er I felt making it implicitly unwrapped resulted in more readable code with very little danger of a crash. The reason that session can't be a let is that when creating the session, the BackgroundDownloader instance sets itself as the session's delegate so the BackgroundDownloader has to exist which means that the initialising of the session has to happen after the super.init() call is made.

So far, we have configured a session for performing background-transfers but don't yet have the ability to download anything so let's add that. A download needs 3 pieces of information:

  1. Remote URL - URL of the asset to be downloaded.
  2. File path URL - URL of where the downloaded asset should be moved to on the local file system.
  3. Completion handler - a closure to be called when the download is complete.

Rather than store these pieces of information about each download in separate arrays, we can create a model class to group these related pieces of information:

typealias ForegroundDownloadCompletionHandler = ((_ result: DataRequestResult) -> Void)

class DownloadItem {
    
    let remoteURL: URL
    let filePathURL: URL
    var foregroundCompletionHandler: ForegroundDownloadCompletionHandler?
    
    // MARK: - Init
    
    init(remoteURL: URL, filePathURL: URL) {
        self.remoteURL = remoteURL
        self.filePathURL = filePathURL
    }
}

An instance of DownloadItem will be created when a new download request is made against BackgroundDownloader. This DownloadItem instance can then be stored in a dictionary to be used once the download is complete:

class BackgroundDownloader: NSObject {
  
    // Omitted properties
    
    private var downloadItems: [URL: DownloadItem] = [:]

    // Omitted methods

    func download(remoteURL: URL, filePathURL: URL, completionHandler: @escaping ForegroundDownloadCompletionHandler) {
        if let downloadItem = downloadItems[remoteURL] {
            print("Already downloading: \(remoteURL)")
            downloadItem.foregroundCompletionHandler = completionHandler
        } else {
            print("Scheduling to download: \(remoteURL)")
            
            let downloadItem = DownloadItem(remoteURL: remoteURL, filePathURL: filePathURL)
            downloadItem.foregroundCompletionHandler = completionHandler
            downloadItems[remoteURL] = downloadItem
            
            let task = session.downloadTask(with: remoteURL)
            task.resume()
        }
    }
}

It's not unusual for an app to request the same download multiple times (think of a user scrolling a tableview up and down) and I wanted to handle this common scenario in the download method. Where the same URL is requested for download while it is already being downloaded, the old completionHandler is switched out for the new completionHandler (This is a fairly naive approach and there are better alternatives such as coalescing closures). When a new URL is requested for download a new DownloadItem instance is created and stored in the downloadItems dictionary with remoteURL used as the key. Next, an URLSessionDownloadTask instance is created and download started.

The URLSessionDownloadDelegate protocol has been mentioned a few times now so lets see how it's implemented:

extension BackgroundDownloader: URLSessionDownloadDelegate {

    func urlSession(_ session: URLSession, downloadTask: URLSessionDownloadTask, didFinishDownloadingTo location: URL) {
        guard let originalRequestURL = downloadTask.originalRequest?.url, let downloadItem = downloadItems[originalRequestURL] else {
            return
        }

        print("Downloaded: \(downloadItem.remoteURL)")

        do {
            try FileManager.default.moveItem(at: location, to: downloadItem.filePathURL)

            downloadItem.foregroundCompletionHandler?(.success(downloadItem.filePathURL))
        } catch {
            downloadItem.foregroundCompletionHandler?(.failure(APIError.invalidData))
        }

       downloadItems[originalRequestURL] = nil
    }
}

As it turns out in this example, only one delegate method is needed.

If the download has been a success, the asset is moved from its temporary location to its permanent location on the file system. Then the completionHandler is triggered with the success case of the DataRequestResult enum containing the permanent location URL.

If the download has failed the completionHandler is triggered with the failure case containing an error.

(This method is deliberately kept as simple as possible but you can easily imagine how it and DownloadItem could be extended to e.g. update Core Data or perform image manipulation)

Finally we have been using DataRequestResult enum as a generic result enum so lets look at as well:

enum DataRequestResult<T> {
    case success(T)
    case failure(Error)
}

And that's all that's required for performing foreground transfers using a background configuration.

If you want to see this version of BackgroundDownloader running, checkout out the foreground_only_transfers branch on the example project - to run this you will need to follow the instructions below.

Working behind the users back

If you run the app in the foreground you will notice that everything works as expected and the app is populated with downloaded assets however as soon as the app is put into the background, a stream of errors will begin to appear in the Xcode console, similar to:

completion-handler-never-called-error

So far BackgroundDownloader only works when the app is in the foreground, let's extend it to support downloads when the app enters a suspended/backgrounded state.

When an app is in a suspended/backgrounded state and a download is completed, iOS will wake the app up, call the app-delegate and pass in a closure to be triggered once the post download actions have been completed. So in order to support background-transfers, we will need to implemented the expected app-delegate method and then pass the above-mentioned closure to BackgroundDownloader which will trigger it once the downloaded asset has been moved to its final location.

class AppDelegate: UIResponder, UIApplicationDelegate, UISplitViewControllerDelegate { 

    // Omitted properties and methods

    func application(_ application: UIApplication, handleEventsForBackgroundURLSession identifier: String, completionHandler: @escaping () -> Void) {
      BackgroundDownloader.shared.backgroundCompletionHandler = completionHandler
    }
}

As the completionHandler closure is passed to the shared BackgroundDownloader instance we need to add a new property to that class:

var backgroundCompletionHandler: (() -> Void)?

All that's required is to trigger the backgroundCompletionHandler when post-processing of the download is complete. When a download completes, one of URLSessionDelegate methods is triggered:

extension BackgroundDownloader: URLSessionDelegate {

    func urlSessionDidFinishEvents(forBackgroundURLSession session: URLSession) {
        DispatchQueue.main.async {
            self.backgroundCompletionHandler?()
            self.backgroundCompletionHandler = nil
        }
    }
}

It's important to trigger this closure as soon as the app is finished with processing the completed download as this instructs iOS to take a new snapshot of the app's UI (for the app switcher preview) and allows iOS to move onto the next task/app.

In order to better see the downloads happening in the background you may want to add a slight delay before the download actually starts so as to allow you to more leisurely put the app into the background:

task.earliestBeginDate = Date().addingTimeInterval(5)

And that's all the changes required to continue network requests when the app is suspended/backgrounded.

If you want to see this version of BackgroundDownloader running, checkout out the foreground_and_suspended_transfers branch on the example project - to run this you will need to follow the instructions below.

Coming back from the dead 🧟

After an app has been terminated, all of the memory which was allocated to it is freed. This means that when iOS relaunches the app to complete a background-transfer we don't have the details in memory that we can use to determine how that background-transfer should be completed. Instead, these details need to be persisted in the file system between app executions. In our example, the only information that we really need to persist is filePathURL (as you will see the remoteURL is also persisted but this is primarily for informational purposes) so introducing something like Core Data to handle this would be overkill. Instead, let's stick with KISS principles and take advantage of the Codable protocol. By conforming DownloadItem to Codeable we can encode that DownloadItem object into its Data representation which can then be stored in User Defaults. The only issue with this approach is that the foregroundCompletionHandler property on DownloadItem can't conform to Codable, so the DownloadItem implementation of Codable will be slightly more involved than usual. Thankfully Codable has an easy way to declare which properties should and shouldn't be encoded.

class DownloadItem: Codable {

    // Properties omitted

    private enum CodingKeys: String, CodingKey {
        case remoteURL
        case filePathURL
    }

    // Methods omitted
}

With these changes, the DownloadItem class now has a dual nature:

  1. Freshly created.
  2. Restored from User Defaults.

This presents a problem with determining which class should be responsible for handling this dual nature - knowing how to save into, load from and clean up User Defaults. I toyed with placing this responsibility in DownloadItem but wanted to keep this class as simple as possible which ruled that approach out. Then I moved onto BackgroundDownloader but again felt that it wasn't the correct class for handling this - I didn't want to expose that some DownloadItem instances were freshly created while some had been restored from User Defaults to BackgroundDownloader whose primary responsibility centred on downloading rather than persistence. In the end, I drew inspiration from Core Data and decided to create a downloading context.

This context class needs to handle 3 different tasks:

  1. Loading.
  2. Saving.
  3. Deleting.

Lets start building this BackgroundDownloaderContext class:

class BackgroundDownloaderContext {

    private var inMemoryDownloadItems: [URL: DownloadItem] = [:]
    private let userDefaults = UserDefaults.standard

}

Like before, a simple dictionary is being used as an in-memory store which will hold all active DownloadItem instances.

When it comes to loading/retrieving an existing DownloadItem instance, this can be from two possible stores:

  1. In-memory.
  2. User Defaults.

Again, we don't want to expose these details so let's add a load method to BackgroundDownloaderContext that can load from either of the two possible stores:

class BackgroundDownloaderContext {

    // Omitted properties

    func loadDownloadItem(withURL url: URL) -> DownloadItem? {
        if let downloadItem = inMemoryDownloadItems[url] {
            return downloadItem
        } else if let downloadItem = loadDownloadItemFromStorage(withURL: url) {
             inMemoryDownloadItems[downloadItem.remoteURL] = downloadItem

            return downloadItem
        }

        return nil
    }

    private func loadDownloadItemFromStorage(withURL url: URL) -> DownloadItem? {
        guard let encodedData = userDefaults.object(forKey: url.path) as? Data else {
            return nil
        }

        let downloadItem = try? JSONDecoder().decode(DownloadItem.self, from: encodedData)
        return downloadItem
    }
}

First, the in-memory store is checked for an existing DownloadItem using the network request's URL as the key. If an existing DownloadItem instance isn't found in the in-memory store, the User Defaults store is checked. If an object is found in User Defaults, an attempt is made to decode it from its raw Data representation to a new DownloadItem instance. As you will have spotted, decoding can potentially throw an exception, if an exception is thrown it will be silently ignored due to the try? statement. I chose this approach as an error during decoding wouldn't be repairable, instead just returning nil would suffice.

In order to get DownloadItem instances into User Defaults we need to save them:

class BackgroundDownloaderContext { 

    // Omitted properties and methods

    func saveDownloadItem(_ downloadItem: DownloadItem) {
        inMemoryDownloadItems[downloadItem.remoteURL] = downloadItem

        let encodedData = try? JSONEncoder().encode(downloadItem)
        userDefaults.set(encodedData, forKey: downloadItem.remoteURL.path)
        userDefaults.synchronize()
    }
}

In the above code snippet, the DownloadItem instance is first placed into the in-memory store before being saved into User Defaults as a Data object.

Finally when a download is complete the in-memory and User Default objects need to be cleaned away:

class BackgroundDownloaderContext { 

    // Omitted properties and methods

    func deleteDownloadItem(_ downloadItem: DownloadItem) {
      inMemoryDownloadItems[downloadItem.remoteURL] = nil
      userDefaults.removeObject(forKey: downloadItem.remoteURL.path)
      userDefaults.synchronize()
    }
}

By implementing the BackgroundDownloaderContext, the BackgroundDownloader logic can be kept pretty much the same as before despite this significant introduction of new functionality:

class BackgroundDownloader: NSObject {

    // Omitted properties
    
    private let context = BackgroundDownloaderContext()

    // Omitted methods

    func download(remoteURL: URL, filePathURL: URL, completionHandler: @escaping ForegroundDownloadCompletionHandler) {
        if let downloadItem = context.loadDownloadItem(withURL: remoteURL) {
            print("Already downloading: \(remoteURL)")
            downloadItem.foregroundCompletionHandler = completionHandler
        } else {
            print("Scheduling to download: \(remoteURL)")

            let downloadItem = DownloadItem(remoteURL: remoteURL, filePathURL: filePathURL)
            downloadItem.foregroundCompletionHandler = completionHandler
            context.saveDownloadItem(downloadItem)

            let task = session.downloadTask(with: remoteURL)
            task.earliestBeginDate = Date().addingTimeInterval(20) // Added a delay for demonstration purposes only
            task.resume()
        }
    }
}

// Omitted methods

extension BackgroundDownloader: URLSessionDownloadDelegate {

    func urlSession(_ session: URLSession, downloadTask: URLSessionDownloadTask, didFinishDownloadingTo location: URL) {
        guard let originalRequestURL = downloadTask.originalRequest?.url, let downloadItem = context.loadDownloadItem(withURL: originalRequestURL) else {
            return
        }

        print("Downloaded: \(downloadItem.remoteURL)")

        do {
            try FileManager.default.moveItem(at: location, to: downloadItem.filePathURL)

            downloadItem.foregroundCompletionHandler?(.success(downloadItem.filePathURL))
        } catch {
            downloadItem.foregroundCompletionHandler?(.failure(APIError.invalidData))
        }

       context.deleteDownloadItem(downloadItem)
    }
}

A new property is added to hold the BackgroundDownloaderContext instance:

private let context = BackgroundDownloaderContext()

When scheduling a new download, the context is checked to see if an existing DownloadItem already exists:

if let downloadItem = context.loadDownloadItem(withURL: remoteURL) {

And when a new DownloadItem instance is created, that instance is saved it into the context:

let downloadItem = DownloadItem(remoteURL: remoteURL, filePathURL: filePathURL)
downloadItem.foregroundCompletionHandler = completionHandler
context.saveDownloadItem(downloadItem)

Finally when the download is complete, the DownloadItem instance is deleted from the context:

context.deleteDownloadItem(downloadItem)

Again, here we have managed to avoid exposing to the BackgroundDownloader if that downloadItem was purely an in-memory object or if it also had a backing User Defaults entry.

As with the suspended/backgrounded code changes, when the app is awoken from a terminated state the app-delegate is called:

class AppDelegate: UIResponder, UIApplicationDelegate, UISplitViewControllerDelegate { 

    // Omitted properties and methods

    func application(_ application: UIApplication, handleEventsForBackgroundURLSession identifier: String, completionHandler: @escaping () -> Void) {
      BackgroundDownloader.shared.backgroundCompletionHandler = completionHandler
    }
}

This is actually the exact same code as in the suspended/backgrounded example above but I wanted to highlight that when the app is awoken from a terminated state, this method not only passes the completionHandler along but also sets up the BackgroundDownloader so that it can respond to the session delegate calls.

An interesting point around terminating the app is that if you manually force quit the app then iOS will take this as definite confirmation that you are finished with the app and so cancel any scheduled background-transfers. This presents a problem when it comes to testing how our solution handles termination. The easiest way I found to overcome this issue was to add a manual exit call to cause the app to terminate without any user involvement:

class AppDelegate: UIResponder, UIApplicationDelegate, UISplitViewControllerDelegate { 

    // Omitted properties and methods

    func applicationDidEnterBackground(_ application: UIApplication) {
        //Exit app to test restoring app from a terminated state. Comment out to test restoring app from a suspended state.
        DispatchQueue.main.asyncAfter(deadline: .now()) {
            print("App is about to quit")

            if let documentsPath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true).first {
                debugPrint("Gallery assets will be saved to: \(documentsPath)")
            }
            exit(0)
        }
    }
}

In the above code snippet, the file path of the directory that the downloaded assets will end up in is printed to the console and then the app terminates. With this file path, you can then navigate to that directory and see the folder fill up with downloaded assets.

Downloads keep going 🤯

We looked at how with just a few fairly small classes we are able to add support for background-transfers into our apps. As apps become more sophisticated our dependency on functionality driven by network requests is only going to increase with users expecting to be able to push and pull ever-larger quantities of data. With background sessions, we have a powerful weapon in ensuring that we can meet our user's wants without having to trap them in the app for hours at a time.

Congratulations on making it to the end 🎉.


Running the example project

In "Not all downloads are born equal" I used Imgur as an online image database and I've used it again in this post to power the example project. Imgur has a great JSON based API and an extensive library of freely available media. Imgur API, while being free, does require us to register our example project to get a client-id which needs to be sent with each request.

As the example project does not require access to a user's personal data, when registering select the Anonymous usage without user authorization option for Authorization type. At the time of writing, you had to provide a URL in the Authorization callback URL field even for anonymous authentication - I found any URL value would work for this.

After you register, you should be given a unique client-id which will allow you access to Imgur's content. You will need to add your client-id as the value of the clientID property in RequestConfig. After you've added your client-id, you should be able to run the app and see how our background-transfer solution works.

What do you think? Let me know by getting in touch on Twitter - @wibosco