Keeping things going when the user leaves with background transfers

In the past iOS was always about living in the moment - totally focused on whatever the user was attempting to do. This offered the user a great experience however as app developers it restricted what we could do because as soon as the app went into the background, iOS suspended it regardless of what the app was doing. However that's no longer 100% true. As users expectations of what their apps can do has increased, iOS has began to allow some limited background processing. This has been possible because the hardware iOS runs on has got more powerful and energy efficient with each release so the device can execute these additional hidden tasks without negatively affecting the user experience.

One way that iOS has eased restrictions is allowing apps to continue network requests in the background or to use the Apple approved terminology: background transfers. The Background Transfer Service was first introduced in iOS 7 so it's not a recent change but it's such a powerful tool that I really wanted to explore it further and especially show a solution for recovering from a terminated state.

Background sessions

When Apple introduced the URLSession suite of classes, it allowed the creation of mulitple network sessions, each with their own configurations. This allowed for easily differentiating between the various types of network requests that an app has to make. The main session configurations are:

  • default - allowing for all types of network requests in the foreground
  • ephemeral - similar to default but more forgetful (doesn’t write caches, cookies, or credentials to disk)
  • background - allowing for uploading and downloading content even when app isn't running

(Of course it's possible to further differentiate sessions within those configurations by adjusting the properties on each session)

At the time URLSession was introduced I was building a very media heavy app and the promise of being being able to continue to upload or download content when the app wasn't running by using a background session was very appealing and I embraced this new functionality with zeal. This post is a look at how I used the opportunity that background sessions presented. But before we get too deep into how to use background sessions, let's take a little peek under the hood of how background sessions work 🕵️.

When scheduling a network request on a background session, that request is being handed off to the nsurlsessiond daemon which runs as a separate process from the app. As this daemon lives outside of the lifecycle of the app if the app is suspended or killed the network request will continue unaffected - this is the major difference from scheduling a network request on a default or ephemeral session where the download is very much tied to the lifecycle of the app. Once a background transfer is complete, iOS then wakes the app up (in the background) and allows the app to finish that download. This waking up happens even if the app has been terminated.

You're probably thinking:

"That sounds pretty amazing! Why isn't this the default behaviour?"

Well, the more background processing that the device has to support the quicker the battery will drain and more bandwidth that will be consumed - all of which will lead to unhappier users for Apple. So while Apple understands that background processing is needed, it wants to ensure that as app developers we use it responsibly as using it comes with the cost described above. Also from a developer POV supporting background transfers requires a bit more programming than a foreground-only session. Rather than the simple closure based API that foreground-only sessions support, to use background sessions the developer needs to conform to various different protocols. This additional knowledge (as we shall see) while not hugely taxing does raise the bar on what is required to be known in order to make a network request. So in true Apple fashion, they have opt'd to not only keep it simple for the users but also for us developers.

Now that the hood is well and truly peeked under, lets get back to how to use background sessions to provide a better experience for our users.

Scenarios

Any background transfer solution should be able to handle 3 different transfer scenarios:

  1. Foreground - when the app is open, the transfer should behave the same way as if the session was using the default configuration.
  2. Suspended/Backgrounded - when the app is in a suspended state and the asset is downloaded, the app should be able to be woken up to finish transfer.
  3. Terminated - when the app has been terminated by iOS, the app should be able to be relaunched to finish that transfer.

In the rest of this post, we will build a download system that will look to handle the 3 scenarios described above. This post will gradually build up to a working example however if you're on a tight deadline and/or there is murderous look creeping into your manager's eyes 😡, then head on over to the completed example and take a look at BackgroundDownloader, BackgroundDownloaderContext, BackgroundDownloadItem and AppDelegate to see how things end up - in order to run this you will need to follow the instructions below.

(This post will focus solely on downloading background transfers as it's much easier to find services that allow us to retrieve content from their system than services that allow us to upload content)

Let's get downloading

First things first, in order to support background transfer we need to grant permission for Background Modes. This is done by opening the Capabilities tab in the target and switching the Background Modes toggle to ON.

In order to build this example we will need to encapsulate our network requests in their own class - BackgroundDownloader. This class will be responsible for configuring our background session, downloading the requested asset and responding to any needed delegate callbacks. When supporting background transfers we need to ensure that a network request is always handled on the same session (not necessarily the same instance of URLSession but a session with the same identifier) and as each session needs to have a unique session identifier, I've decided to make BackgroundDownloader a singleton.

Lets build the skeleton of the BackgroundDownloader class:

class BackgroundDownloader: NSObject {
    
    private let fileManager = FileManager.default
    private var session: URLSession!

    // MARK: - Singleton

    static let shared = BackgroundDownloader()

    // MARK: - Init

    private override init() {
        super.init()

        let configuration = URLSessionConfiguration.background(withIdentifier: "background.download.session")
        session = URLSession(configuration: configuration, delegate: self, delegateQueue: nil)
    }
}

In the above code snippet an instance of URLSessionConfiguration is created which is then passed a URLSession instance during its initialisation. As mentioned above, the identifier that the configuration accepts needs to be unique - if multiple sessions have the same identifier, iOS has the rather alarming warning in its documentation:

"You must create exactly one session per identifier (specified when you create the configuration object). The behavior of multiple sessions sharing the same identifier is undefined."

undefined doesn't sound like a good state for the app to get into 😬 so lets avoid it. As BackgroundDownloader is a singleton I've ensured that only one session will have this identifier but what's also important to note is that this identifier is a staticly valued string rather than say generating it using UUID().uuidString - this will ensure that even between app executions the session can always be recreated (with the same identifier) and any background transfers scheduled against that older session (from the app that was terminated) can be completed on the newer session because they share the same identifier (more on how to do this shortly).

An interesting difference between background sessions and other types of sessions is that with a background we can't use the completionHandler methods when creating a URLSessionDownloadTask task (doing so won't result in the compiler throwing an error but the app will throw an exception at run time). Instead of using the closure approach, BackgroundDownloader will need to implement the URLSessionDownloadDelegate protocol (and eventually URLSessionDelegate) - this need to implement these delegates is the reason that BackgroundDownloader is a subclass of NSObject.

The more eagle-eyed among you will have noticed that the session property is implicitly unwrapped. Normally I try and avoid implicitly unwrapped properties but as session is a private property and is set in the init'er I felt making it implicitly unwrapped resulted in more readable code with very little danger of a crash. The reason that session can't be a let is that when creating the session, the BackgroundDownloader instance sets itself as the session's delegate so the BackgroundDownloader has to exist which means that the initialising of the session has to happen after the super.init() call is made.

So far, we have configured a session for performing background transfers but don't yet have the ability to download anything so lets add that in. A download needs 3 pieces of information:

  1. Remote URL - URL of the asset to be downloaded.
  2. File path URL - URL of where the downloaded asset should be moved to on the local file system.
  3. Completion handler - a closure to be called when the download is complete.

Rather than store these pieces of information about each download in separate arrays, we can create a model class to group these related pieces of information:

typealias ForegroundDownloadCompletionHandler = ((_ result: DataRequestResult) -> Void)

class DownloadItem {
    
    let remoteURL: URL
    let filePathURL: URL
    var foregroundCompletionHandler: ForegroundDownloadCompletionHandler?
    
    // MARK: - Init
    
    init(remoteURL: URL, filePathURL: URL) {
        self.remoteURL = remoteURL
        self.filePathURL = filePathURL
    }
}

An instance of DownloadItem will be created when a new download request is made against BackgroundDownloader. This DownloadItem instance can then be stored in a dictionary to be used once the download is complete:

private var downloadItems: [URL: DownloadItem] = [:]

// Other properties and methods omitted

func download(remoteURL: URL, filePathURL: URL, completionHandler: @escaping ForegroundDownloadCompletionHandler) {
    if let downloadItem = downloadItems[remoteURL] {
        print("Already downloading: \(remoteURL)")
        downloadItem.foregroundCompletionHandler = completionHandler
    } else {
        print("Scheduling to download: \(remoteURL)")
        
        let downloadItem = DownloadItem(remoteURL: remoteURL, filePathURL: filePathURL)
        downloadItem.foregroundCompletionHandler = completionHandler
        downloadItems[remoteURL] = downloadItem
        
        let task = session.downloadTask(with: remoteURL)
        task.resume()
    }
}

It's not unusual for an app to request the same download multiple times (think of a user scrolling a tableview up and down) and I wanted to handle this common scenario in the download method. Where the same url is requested for download while it is already being downloaded, the old completionHandler is switched out for the new completionHandler (This is a fairly naive approach and there are better alternatives such as coalescing closures). When a new url is requested for download a new DownloadItem instance is created and stored in the downloadItems dictionary with its unique remoteURL value as the key. Next an URLSessionDownloadTask instance is created and download started.

The URLSessionDownloadDelegate protocol has been mentioned a few times now so lets see how it's implemented:

extension BackgroundDownloader: URLSessionDownloadDelegate {

    func urlSession(_ session: URLSession, downloadTask: URLSessionDownloadTask, didFinishDownloadingTo location: URL) {
        guard let originalRequestURL = downloadTask.originalRequest?.url, let downloadItem = downloadItems[originalRequestURL] else {
            return
        }

        print("Downloaded: \(downloadItem.remoteURL)")

        do {
            try fileManager.moveItem(at: location, to: downloadItem.filePathURL)

            downloadItem.foregroundCompletionHandler?(.success(downloadItem.filePathURL))
        } catch {
            downloadItem.foregroundCompletionHandler?(.failure(APIError.invalidData))
        }

       downloadItems[originalRequestURL] = nil
    }
}

As it turns out in this example only one delegate method is needed.

If the download has been a success the asset is moved from its temporary location to its permanent location on the file system. Then the completionHandler is triggered with the success case of the DataRequestResult enum containing the permanent location url.

If the download has failed the completionHandler is triggered with the failure case containing an error.

(This method is deliberately kept as simple as possible but you can easily imagine how it and DownloadItem could be extended to e.g. update Core Data or perform image manipulation)

Finally we have been using DataRequestResult enum as a generic result enum so lets look at as well:

enum DataRequestResult {
    case success(T)
    case failure(Error)
}

And that's all thats required for performing foreground transfers using a background configuration.

If you want to see this version of BackgroundDownloader running, checkout out the foreground_only_transfers branch on the example project - in order to run this you will need to follow the instructions below.

Working behind the users back

If you run the app in the foreground you will notice that everything works as expected and the app is populated with downloaded assets however as soon as the app is put into the background, a stream of errors will begin to appear in the Xcode console, similar to:

completion-handler-never-called-error

So far BackgroundDownloader only works when the app is in the foreground, lets extend it to support downloads when the app enters a suspended/backgrounded state.

When an app is in a suspended/backgrounded state and a download is completed, iOS will wake the app up, call the app-delegate and pass in a closure to be triggered once the post download actions have been completed. So in order to support background transfers we will need to add a method to app-delegate and then pass the above mentioned closure to BackgroundDownloader which will then trigger it once the downloaded asset has been moved to its final location.

func application(_ application: UIApplication, handleEventsForBackgroundURLSession identifier: String, completionHandler: @escaping () -> Void) {
    BackgroundDownloader.shared.backgroundCompletionHandler = completionHandler
}

The above method is added into the app-delegate and it's here that we see the need for BackgroundDownloader to be a singleton in order to get access to the same session from both the GalleryAssetDataManager (no need to scroll back up the page, we haven't seen this class yet - it's in the example project) and AppDelegate. The completionHandler closure is passed directly from iOS to our app. It's important to trigger this closure once we are finished with processing the completed download as this instructs iOS to take a new snapshot of the app's UI (for the app switcher preview) and allows iOS to move onto the next task/app as quickly as possible. As the completionHandler closure is passed to the shared BackgroundDownloader instance we need to add a new property to that class:

var backgroundCompletionHandler: (() -> Void)?

All that's required is to trigger the backgroundCompletionHandler when post-processing of the download is complete. When a download completes, one of URLSessionDelegate methods is triggered:

extension BackgroundDownloader: URLSessionDelegate {

    func urlSessionDidFinishEvents(forBackgroundURLSession session: URLSession) {
        DispatchQueue.main.async {
            self.backgroundCompletionHandler?()
            self.backgroundCompletionHandler = nil
        }
    }
}

It's important to nil out the completion handler once we call it to ensure that it's not accidentally triggered again.

In order to better see the downloads happening in the background you may want to add a slight delay before the download actually starts, this will give you the chance to put the app into the background:

task.earliestBeginDate = Date().addingTimeInterval(5)

And that's all the changes required to continue network requests when the app is suspended/backgrounded.

If you want to see this version of BackgroundDownloader running, checkout out the foreground_and_suspended_transfers branch on the example project - in order to run this you will need to follow the instructions below

Coming back from the dead 🧟

When an app is terminated, all of the memory which was allocated to it is freed. This means that when iOS relaunches the app to complete a background transfer we don't have the details in memory that we can use to determine how that background transfer should be completed. Instead these details need to be persisted in the file system between app executions. The only information that we really need to persist is filePathURL (as you will see the remoteURL is also persisted but this is primarily for informational purposes) so introducing something like Core Data to handle this would be overkill. Instead lets stick with KISS principles and take advantage of the fairly new Codable protocol. By conforming DownloadItem to Codeable we can encode that DownloadItem object into its Data representation, this data object can then be stored in User Defaults. The only issue with this approach is that the foregroundCompletionHandler property on DownloadItem can't conform to Codable, so the DownloadItem implementation of Codable will be slightly more involved than normal. Thankfully Codable has an easy way to declare which properties should and shouldn't be achieved.

typealias ForegroundDownloadCompletionHandler = ((_ result: DataRequestResult) -> Void)

class DownloadItem: Codable {

    let remoteURL: URL
    let filePathURL: URL
    var foregroundCompletionHandler: ForegroundDownloadCompletionHandler?

    private enum CodingKeys: String, CodingKey {
        case remoteURL
        case filePathURL
    }

    // MARK: - Init

    init(remoteURL: URL, filePathURL: URL) {
        self.remoteURL = remoteURL
        self.filePathURL = filePathURL
    }
}

With these changes, DownloadItem class now has a dual nature - freshly created and restored from User Defaults. This presents a problem with determining which class should be responsible for handling this dual nature - knowing how to save into, load from and cleanup User Defaults. I toyed with placing this responsibility in DownloadItem but really wanted to keep this class as simple as possible which ruled that approach out. Then I moved onto BackgroundDownloader but again felt that it wasn't the correct class for handling this - I didn't want to expose that some DownloadItem instances were freshly created while some had been restored from User Defaults to BackgroundDownloader whose primary responsibility centred on downloading rather than persistence. In the end I drew inspiration from Core Data and decided to create my own context (don't worry it's not actually related to NSManagedObjectContext in any way other than a few shared ideas). This context class needs to handle 3 different tasks:

  • loading
  • saving
  • deleting

Lets start building this BackgroundDownloaderContext class:

class BackgroundDownloaderContext {

    private var inMemoryDownloadItems: [URL: DownloadItem] = [:]
    private let userDefaults = UserDefaults.standard

}

Like before, a simple dictionary is being used as an in-memory store which will hold all active DownloadItem instances.

When it comes to loading/retrieving an existing DownloadItem instance, this can be from two possible stores - in-memory or User Defaults:

func loadDownloadItem(withURL url: URL) -> DownloadItem? {
    if let downloadItem = inMemoryDownloadItems[url] {
        return downloadItem
    } else if let downloadItem = loadDownloadItemFromStorage(withURL: url) {
         inMemoryDownloadItems[downloadItem.remoteURL] = downloadItem

        return downloadItem
    }

    return nil
}

private func loadDownloadItemFromStorage(withURL url: URL) -> DownloadItem? {
    guard let encodedData = userDefaults.object(forKey: url.path) as? Data else {
        return nil
    }

    let downloadItem = try? JSONDecoder().decode(DownloadItem.self, from: encodedData)
    return downloadItem
}

First the in-memory store is checked for an existing DownloadItem using the network request's url as the key. If an existing DownloadItem instance isn't found in the in-memory store, the User Defaults store is checked. The network request's url is used as the key for User Defaults, if an object is found, an attempt is made to decode it from its raw Data representation to a new DownloadItem instance. As you will have spotted, decoding can potentially throw an exception, if an exception is thrown it will be silently ignored due to the try? statement. I chose this approach as an error during decoding wouldn't be repairable, instead just returning nil would suffice.

In order to get DownloadItem instances into User Defaults we need to save them:

func saveDownloadItem(_ downloadItem: DownloadItem) {
    inMemoryDownloadItems[downloadItem.remoteURL] = downloadItem

    let encodedData = try? JSONEncoder().encode(downloadItem)
    userDefaults.set(encodedData, forKey: downloadItem.remoteURL.path)
    userDefaults.synchronize()
}

In the above code snippet, the DownloadItem instance is first placed into the in-memory store before being saved into User Defaults as a Data object.

Finally when a download is complete the in-memory and User Default objects need to be cleaned away:

func deleteDownloadItem(_ downloadItem: DownloadItem) {
    inMemoryDownloadItems[downloadItem.remoteURL] = nil
    userDefaults.removeObject(forKey: downloadItem.remoteURL.path)
    userDefaults.synchronize()
}

By implementing the BackgroundDownloaderContext, the BackgroundDownloader logic can be kept pretty much the same as before despite this significant introduction of new functionality:

class BackgroundDownloader: NSObject {

    var backgroundCompletionHandler: (() -> Void)?

    private let fileManager = FileManager.default
    private let context = BackgroundDownloaderContext()
    private var session: URLSession!

    // MARK: - Singleton

    static let shared = BackgroundDownloader()

    // MARK: - Init

    private override init() {
        super.init()

        let configuration = URLSessionConfiguration.background(withIdentifier: "background.download.session")
        session = URLSession(configuration: configuration, delegate: self, delegateQueue: nil)
    }

    // MARK: - Download

    func download(remoteURL: URL, filePathURL: URL, completionHandler: @escaping ForegroundDownloadCompletionHandler) {
        if let downloadItem = context.loadDownloadItem(withURL: remoteURL) {
            print("Already downloading: \(remoteURL)")
            downloadItem.foregroundCompletionHandler = completionHandler
        } else {
            print("Scheduling to download: \(remoteURL)")

            let downloadItem = DownloadItem(remoteURL: remoteURL, filePathURL: filePathURL)
            downloadItem.foregroundCompletionHandler = completionHandler
            context.saveDownloadItem(downloadItem)

            let task = session.downloadTask(with: remoteURL)
            task.earliestBeginDate = Date().addingTimeInterval(20) // Added a delay for demonstration purposes only
            task.resume()
        }
    }
}

// MARK: - URLSessionDelegate

extension BackgroundDownloader: URLSessionDelegate {

    func urlSessionDidFinishEvents(forBackgroundURLSession session: URLSession) {
        DispatchQueue.main.async {
            self.backgroundCompletionHandler?()
            self.backgroundCompletionHandler = nil
        }
    }
}

// MARK: - URLSessionDownloadDelegate

extension BackgroundDownloader: URLSessionDownloadDelegate {

    func urlSession(_ session: URLSession, downloadTask: URLSessionDownloadTask, didFinishDownloadingTo location: URL) {
        guard let originalRequestURL = downloadTask.originalRequest?.url, let downloadItem = context.loadDownloadItem(withURL: originalRequestURL) else {
            return
        }

        print("Downloaded: \(downloadItem.remoteURL)")

        do {
            try fileManager.moveItem(at: location, to: downloadItem.filePathURL)

            downloadItem.foregroundCompletionHandler?(.success(downloadItem.filePathURL))
        } catch {
            downloadItem.foregroundCompletionHandler?(.failure(APIError.invalidData))
        }

       context.deleteDownloadItem(downloadItem)
    }
}

A new property is added to hold the BackgroundDownloaderContext instance:

private let context = BackgroundDownloaderContext()

When scheduling a new download, the context is checked to see if an existing DownloadItem already exists:

if let downloadItem = context.loadDownloadItem(withURL: remoteURL) {

And when a new DownloadItem instance is created, that instance is saved it into the context:

let downloadItem = DownloadItem(remoteURL: remoteURL, filePathURL: filePathURL)
downloadItem.foregroundCompletionHandler = completionHandler
context.saveDownloadItem(downloadItem)

Finally when the download is complete, the DownloadItem instance is deleted from the context:

context.deleteDownloadItem(downloadItem)

Again, here we have managed to avoid exposing to the BackgroundDownloader if that downloadItem was purely an in-memory object or if it also had a backing User Defaults entry.

As with the suspended/backgrounded code changes, when the app is awoken from a terminated state the app-delegate is called:

func application(_ application: UIApplication, handleEventsForBackgroundURLSession identifier: String, completionHandler: @escaping () -> Void) {
    BackgroundDownloader.shared.backgroundCompletionHandler = completionHandler
}

This is actually the exact same code as in the suspended/backgrounded example above but I wanted to highlight that when the app is awoken from a terminated state, this method not only passes the completionHandler along but also sets up the BackgroundDownloader so that it can respond to the session delegate calls.

An interesting point around terminating the app is that if you manually force quit the app then iOS will take this as definite confirmation that you are finished with the app and so cancel any scheduled background transfers. This presents a problem when it comes to testing how our solution handles termination. The easiest way I found to overcome this issue was to add a manual exit call to cause the app to terminate without any user involvement:

func applicationDidEnterBackground(_ application: UIApplication) {
    //Exit app to test restoring app from a terminated state. Comment out to test restoring app from a suspended state.
    DispatchQueue.main.asyncAfter(deadline: .now()) {
        print("App is about to quit")

        if let documentsPath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true).first {
            debugPrint("Gallery assets will be saved to: \(documentsPath)")
        }
        exit(0)
    }
}

In the above code snippet, the file path of the directory that the downloaded assets will end up in is printed to the console and then the app terminates. With this file path you can then navigate to that directory and after ~20 seconds, see the folder fill up with downloaded assets.

Downloads keep going 🤯

We looked at how with just a few fairly small classes we are able to add support for background transfers into our apps. As the apps we develop become more sophisticated the scale of network requests is only going to increase with users expecting to be able to push and pull ever larger files. With background transfers we have a powerful weapon in ensuring that we can meet our user's wants without having to trap them in the app for hours at a time.

Congratulations on making it to the end 🎉.


Running the example project

In "Not all downloads are born equal" I used Imgur as an online image database and I've used it again in this post to power the example project. Imgur has a great JSON based API and a large library of freely available media. Imgur API, while being free, does require us to register our example project with them in order to get a client-id which we will then send with each request.

In order to run the example project, you will need to register.

As the example project does not require access to a user's personal data, when registering select the Anonymous usage without user authorization option for Authorization type. At the time of writing you had to provide a value in the Authorization callback URL field even for anonymous authentication - I found any url value would work for this.

After you register, you should be given a unique client-id which will allow you access to Imgur's content. Now that you have the client-id, head on over to my GitHub account where you can download the completed example project. You will need to add your client-id as the value of the clientID property which is found in the RequestConfig class. After you've added your client-id, you should be able to run the app and see how our background transfer solution works.

What do you think? Let me know by getting in touch on Twitter - @wibosco