LanguageModelSession CancellationError: Causes and Safe Handling

LanguageModelSession is Apple's Swift concurrency-native API for running on-device language model inference through Apple Foundation Models. It slots cleanly into async/await code, streams tokens through AsyncSequence, and runs entirely on-device via the Neural Engine — no cloud, no API key, no data leaving the device.

But there's one error that catches almost every team on first integration: CancellationError.

It propagates through your task hierarchy, and if you fold it into a generic error path, the app can show a false failure state or discard a partial response without explaining why. This article explains when to expect cancellation and how to handle it without destabilising your UI.

---

What LanguageModelSession Actually Does Under the Hood

Before getting into cancellation, it helps to understand the execution model.

When you call session.respond(to:) or iterate over a streaming response, you're suspending a Swift Task. The model runs inference on the Neural Engine asynchronously, and the session holds a reference to that in-flight work.

Because the session is built on Swift structured concurrency, it participates fully in task cancellation. That's a feature. It means you can cancel expensive inference mid-stream without leaking resources. But it also means cancellation can arrive from directions you might not anticipate.

---

Common Paths That Produce Cancellation

1. The Parent Task Was Cancelled

This is the most common cause. Launch a Task from a SwiftUI view, and if the view disappears before inference completes, the task gets cancelled. The CancellationError propagates up through your await call on session.respond(to:).

.task {
    do {
        let response = try await session.respond(to: prompt)
        self.output = response.content
    } catch {
        // If the view disappears mid-inference, this catches CancellationError
        self.output = ""
    }
}

The .task modifier in SwiftUI cancels automatically on view disappearance — that's the right behaviour. The problem is when teams don't distinguish between a cancellation and a real error.

2. You Called cancel() on the Task Explicitly

If you give the user a stop button and call task.cancel(), the session throws CancellationError at the next suspension point. This is expected and correct. The challenge is handling it gracefully in the UI rather than treating it as a failure state.

3. The Owning Feature Replaced Its In-Flight Task

Search, autocomplete, and rewrite features often cancel the previous request when a new prompt arrives. The cancellation still originates from Task.cancel(), but ownership can make it look unrelated to the call that observes the error. Keep the current task in one well-defined owner and make replacement explicit.

The session should generally live at the scope of the conversation or feature so its transcript and lifecycle are predictable. That is an ownership rule, not a claim that ordinary local-variable deallocation is a documented source of CancellationError.

---

How to Handle It Without Crashing

Distinguish CancellationError from Other Errors

The most important pattern here is catching CancellationError separately from everything else. Treating it as a failure means showing error UI to someone who just navigated away or tapped stop — which is wrong.

do {
    let response = try await session.respond(to: prompt)
    await MainActor.run { self.output = response.content }
} catch is CancellationError {
    // Inference was cancelled intentionally. No error state needed.
    await MainActor.run { self.state = .idle }
} catch {
    // A real error. Surface it.
    await MainActor.run { self.state = .failed(error) }
}

This is the minimum viable pattern. Anything less and you're either swallowing real errors or showing false failure messages.

Check for Cancellation Before Starting Inference

If your task might have been cancelled before it even reaches session.respond(to:), check first:

try Task.checkCancellation()
let response = try await session.respond(to: prompt)

This matters most when you're queuing multiple inference requests. If a request is already stale by the time it reaches the front of the queue, cancel it early rather than burning Neural Engine time on a result nobody needs.

Handle Cancellation in Streaming Responses

Streaming via AsyncSequence needs its own handling. The for await loop exits cleanly on cancellation, but you need to make sure your partial state is consistent:

var accumulated = ""
do {
    for try await partial in session.streamResponse(to: prompt) {
        try Task.checkCancellation()
        accumulated += partial
        await MainActor.run { self.streamingOutput = accumulated }
    }
} catch is CancellationError {
    // Stream was cancelled. accumulated may be partial.
    await MainActor.run { self.streamingOutput = "" }
} catch {
    await MainActor.run { self.state = .failed(error) }
}

Whether you keep partial output depends on your product. For a writing assistant, partial text might be useful. For a classification result, it's probably meaningless. Make that decision explicitly rather than leaving it to chance.

---

Structuring the Session for Predictable Lifecycle

The session's lifecycle should match the inference task's lifecycle. A few patterns that hold up well in production:

Actor-isolated session: Wrap LanguageModelSession in an actor to serialise access and ensure the session lives long enough for inference to complete.

actor InferenceEngine {
    private let session = LanguageModelSession()
    private var currentTask: Task<Void, Never>?

    func run(prompt: String, onResult: @escaping (String) -> Void) {
        currentTask?.cancel()
        currentTask = Task {
            do {
                let response = try await session.respond(to: prompt)
                guard !Task.isCancelled else { return }
                onResult(response.content)
            } catch is CancellationError {
                // Previous request was superseded. Normal.
            } catch {
                // Surface real errors.
            }
        }
    }
}

This pattern handles the "user types fast" case automatically. Each new prompt cancels the previous inference. CancellationError from the cancelled task is caught and discarded. Only the latest result surfaces.

One session per conversation context: LanguageModelSession maintains conversation context. Creating a new session per message loses that context and pays initialisation cost on every call. Create the session once per conversation and reuse it.

---

What Not to Do

A few patterns that reliably cause problems:

Catching all errors with a generic catch block. This buries CancellationError inside a generic failure state and makes debugging harder. Always catch CancellationError first.

Creating a new session for every prompt. This discards conversational context and makes task ownership harder to reason about. Keep one session per conversation or feature context when stateful behavior is intended.

Ignoring Task.isCancelled after an await. Cancellation doesn't always throw immediately. After any await, check Task.isCancelled if you're doing work that shouldn't run on a stale task.

Retrying on CancellationError. Cancellation is intentional. Retrying adds latency and burns Neural Engine resources. Only retry on real errors.

---

Where This Fits in a Larger On-Device AI Architecture

LanguageModelSession cancellation handling is one layer of a production on-device AI integration. The session manages inference lifecycle. Above it, you need a view model or actor managing task lifecycle. Above that, you need UI state that correctly reflects idle, loading, streaming, cancelled, and failed as distinct cases — not a binary success/error toggle.

If you're building this from scratch, the Core ML integration checklist for 2026 covers the full integration surface, including model loading, memory pressure handling, and Neural Engine availability checks that sit alongside session management.

For a production example of on-device AI handling in a real app, the Echo Survival AI case study shows how this architecture holds up under real usage patterns, including offline scenarios where inference must complete without any network fallback.

The broader integration patterns — including how LanguageModelSession fits alongside Core ML pipelines — are covered in Core ML 8 integration patterns for production iOS.

---

The Short Version

CancellationError from LanguageModelSession is normally structured concurrency working correctly. The direct causes are task cancellation inherited from a parent or requested explicitly by your code. Catch cancellation separately, treat it as an intentional stop rather than a failure, and keep task ownership and session scope clear.

Get those three things right and your on-device AI integration handles cancellation without crashing, without false error states, and without wasting Neural Engine time on stale requests.

---

FAQs

What is LanguageModelSession in iOS? LanguageModelSession is Apple's API for running on-device language model inference using Apple Foundation Models. It integrates with Swift structured concurrency via async/await and AsyncSequence, runs entirely on-device via the Neural Engine, and requires no network connection or cloud dependency.

Why does LanguageModelSession throw CancellationError? Because it participates in Swift structured concurrency. The most common causes are that the parent task was cancelled—for example, when a SwiftUI .task leaves the view hierarchy—or your code explicitly cancelled or replaced the in-flight task.

How do I handle CancellationError from LanguageModelSession without showing an error to the user? Catch it in a separate catch is CancellationError block before your general error handler. Treat it as an intentional stop and reset your UI to an idle state rather than a failure state.

Should I retry inference after a CancellationError? No. Cancellation is intentional. Retrying adds latency and wastes Neural Engine resources. Only retry on genuine errors.

Where should I store LanguageModelSession? Store it at the scope of the conversation or feature—commonly in an actor, service, or observable model—so transcript state and task ownership are explicit. A one-shot local session is still valid when you intentionally need no conversational state.

What is the best architecture for managing multiple concurrent LanguageModelSession requests? An actor-isolated wrapper works well. Store the session as a property on the actor, keep a reference to the current Task, and cancel the previous task before starting a new one. This serialises access, prevents session deallocation, and handles rapid user input without resource leaks.

Does CancellationError affect streaming responses differently than single responses? The error type is the same, but with streaming you also need to decide what to do with partial output. A for try await loop exits cleanly on cancellation, but the accumulated partial string may be incomplete. Decide explicitly whether to keep or discard it based on what your app actually needs.

---

Start a project → /apply