How to Integrate On-Device AI
into Your iOS App Without
Sending Data to the Cloud

Use Core ML when:

• The AI task is classification, detection, segmentation, regression, or prediction
• The feature must run on devices going back to iOS 16 or A12 Bionic
• You have a custom model trained in PyTorch or TensorFlow
• Inference speed must be under 20ms (real-time camera, audio, or typing features)

Use Apple Foundation Models when:

• The AI task involves text: summarization, generation, intent classification, or slot extraction
• The user's device is iPhone 15 Pro or later, or any M-series iPad or Mac
• You need structured output from a language model (typed Swift structs from free text)
• You want generative AI features without bundling a model file or paying API costs

Privacy rule that applies to both:

• Neither framework makes network calls during inference — the data boundary is maintained by design, not by configuration
• If your implementation ever calls a cloud AI API (OpenAI, Anthropic, Google) at inference time, it is no longer on-device AI

Core ML model conversion (coremltools):

• Install coremltools: pip install coremltools
• Trace your PyTorch model: traced = torch.jit.trace(model, example_input)
• Convert: mlmodel = ct.convert(traced, compute_units=ct.ComputeUnit.ALL)
• Quantize to reduce size: ct.optimize.coreml.palettize_weights(mlmodel, config=...) — Int4 palettization reduces a 100MB model to ~25MB with minimal accuracy loss for most vision and NLP architectures
• Save: mlmodel.save('YourModel.mlpackage') and drag into your Xcode project

Apple Foundation Models (no model file needed):

• Add the FoundationModels framework to your target in Xcode
• Add the com.apple.developer.foundation-models entitlement to your .entitlements file
• The OS provides the model — nothing to download, convert, or bundle
• Check availability at runtime before using: LanguageModelSession.availability

Core ML actor pattern:

• Declare an actor (not a class or struct) to hold the loaded MLModel
• Load the model with: self.model = try await MLModel.load(contentsOf:) — this is async and non-blocking
• Expose inference as: func predict(_ input: YourModelInput) async throws -> YourModelOutput
• In your SwiftUI view model, call with: let result = try await inferenceActor.predict(input)
• The Swift concurrency runtime routes the actor method off the main thread automatically

Foundation Models actor pattern:

• Create a LanguageModelSession inside an actor
• Define your output type as a Swift struct conforming to Generable
• For streaming: use session.streamResponse(to: prompt) which returns an AsyncStream
• Surface output to SwiftUI with @Published var streamedText = String() updated from the stream
• Wrap the entire session lifecycle in the actor to prevent concurrent session access

What NOT to do:

• Do not call MLModel.prediction() or LanguageModelSession methods from @MainActor context — this blocks the UI thread
• Do not store MLModel as a global singleton — actor isolation makes concurrency safe
• Do not make network calls inside inference functions — if you find yourself adding URLSession calls, you have left on-device AI

Method 1 — Xcode Network Instrument:

• Open Instruments, select the Network template, attach to your app
• Trigger all AI inference paths: every feature, every input type, every error condition
• Confirm zero connections appear in the Network Instrument timeline during inference
• If any connection fires, identify the source — it is either your code or a third-party SDK

Method 2 — Debug entitlement restriction:

• In a Debug build configuration, add an App Sandbox entitlement that disables outgoing connections
• Any network call in any code path (including SDKs) becomes an immediate runtime error
• This catches accidental cloud calls that silent logging might miss

Privacy Nutrition Label review:

• For on-device inference, you do not need to declare data types sent to third-party AI providers
• Document this explicitly in your internal privacy review: 'AI inference input and output are processed on-device. No data transmitted to any third party during AI feature operation.'
• If you later add a cloud fallback path, this declaration must be updated before the next App Store submission

What to measure:

• Inference latency: time from prediction call to result — target under 15ms for real-time features, under 500ms for user-triggered features
• Model load time: first prediction call after app launch includes model compilation — measure and consider preloading
• Peak memory during model load: models expand in memory during loading; a 20MB model may use 80–120MB RAM at peak
• Neural Engine utilization: Instruments Core ML Instrument shows which compute unit handles each layer — confirm ANE is active, not CPU fallback

What to optimize if targets are missed:

• Inference latency too high: apply 4-bit palettization to reduce memory bandwidth, verify computeUnits=.all, reduce input resolution for vision models
• Memory too high: use MLModelConfiguration.functionNames to load only the prediction function, not the full model interface
• Neural Engine not used: check for unsupported operations in your model architecture — ReLU, Conv2D, and linear layers are ANE-compatible; custom ops may force CPU fallback

Device capability gates:

• Core ML: add a guard that checks ProcessInfo.processInfo.processorCount and device model before enabling the feature — ensure the minimum deployment target actually has A12 Bionic or later
• Foundation Models: always check LanguageModelSession.availability before creating a session — the API returns .available, .unavailable, or a specific reason code
• Show a non-AI fallback experience for users on unsupported hardware — never crash or degrade silently

Feature flag and staged rollout:

• Implement a remote feature flag that enables/disables the AI feature independently of the app version — this lets you disable the feature if a memory or performance issue appears post-launch without waiting for an App Store update
• Deploy to 10% of users first, monitor crash rates (Xcode Organizer or Crashlytics), watch memory pressure in analytics, then expand to 50% and 100%
• Log inference success/failure rate (not inference input or output — that would defeat the privacy purpose) to detect silent failures