Yes, the Apple Watch’s New Audio Intelligence Features Really Are Always Listening to You

During its Sept. 9 keynote, Apple unveiled the Apple Watch Series 12 and the Ultra 4, both featuring technology that passively monitors your conversations throughout the day to provide on-demand summaries. These capabilities fall under the “Audio Intelligence” umbrella, with two primary functions to note:

Live Rewind: By double-pressing the Digital Crown on your Apple Watch, you can instantly pull up a text snippet of the conversation from the previous fifteen seconds.

Siri Recaps: Your watch monitors your discussions and generates an AI-powered summary of your transcribed audio at the end of each day.

While this concept might initially sound like a privacy nightmare, Apple has implemented several safeguards to reassure users. The captured audio is never stored or recorded, nor does it leave your device; it is essentially deleted in real time from local storage. Only a truncated version is sent to Apple Private Cloud Compute servers for AI processing, and the resulting transcript delivered to you is end-to-end encrypted.

Nevertheless, there are valid privacy concerns to consider, particularly because you are effectively subjecting those around you to this technology without their explicit consent.

Series 12 and Ultra 4 watches utilize an S11 chip equipped with a Secure Exclave—a partitioned local storage area inaccessible even to the operating system. When your Apple Watch captures ambient audio, it is routed directly into this protected buffer, where it is encrypted before being sent to a corresponding Secure Enclave on your paired iPhone.

From there, your iPhone performs local transcription using an on-device AI model to generate a summary that is half the length of the original audio, stripping out identifying information and non-essential details. Once the audio data is overwritten, the transcript is transmitted to Apple’s Private Cloud Compute along with generic metadata, such as timestamps and a rough location.

At this stage, the audio data is no longer linked to your Apple ID once it reaches the servers. Apple employs a third-party relay to mask your IP address during the upload to Private Cloud Compute, ensuring that even their own employees cannot identify you via your device. Within the Private Cloud Compute servers, your audio is summarized by Siri AI. This process results in the daily text summaries you receive if you choose to opt into Siri Recap. These summaries are automatically purged every seven days unless you manually save them.

While this may appear highly secure, a potential risk remains: if you decide to save a conversation summaryage and may be saved in applications like Journal or Apple Notes. Although server-level breaches at companies like Apple are uncommon, account-level compromises occur more frequently than many realize; for instance, 900,000 compromised iCloud logins were discovered on a public server following a data leak this past January

If an unauthorized party gains access to your Apple ID or an unsecured iPhone, they could potentially view all your saved conversation summaries in one location. Beyond the risk of a data breach, the broader privacy implications are equally concerning.

Apple has invested significant effort into protecting its customers when using Audio Intelligence, but these safeguards do not extend to other individuals who may be sitting in the same room or across the table from you.

To its credit, Apple has included an audio chime and a brief visual cue on the watch whenever the Live Rewind feature is activated. This is intended to alert those nearby that they are being monitored. Furthermore, Live Rewind does not identify speakers, meaning it will not name or otherwise distinguish individuals in the transcriptions or summaries—yet this does not prevent people from being eavesdropped on without their explicit consent.

Siri Recap’s implementation is arguably even worse. It lacks audio or visual cues to indicate it is active, meaning no one is aware they are being surveilled if you have enabled the feature. Apple’s privacy documentation is notably sparse regarding consent, offering only a vague warning on product pages that advises users to “be mindful of those around you.”

This poses a concern not only for those worried about being spied on, but also for Apple Watch users themselves, particularly in regions where local laws mandate consent before recording audio. Approximately 12 U.S. states have strict all-party consent laws prohibiting undisclosed recordings, and many E.U. countries have similar requirements; additionally, the E.U.’s GDPR prohibits recordings containing personally identifiable information without disclosure.

While using an Apple Watch with Audio Intelligence enabled might seem like it could violate these laws, the device currently operates in a legal grey area. As Apple’s privacy documentation notes:

“Audio Intelligence features do not create audio recordings that can be accessed by the operating system, apps, the user, or Apple. There is no recording to share, forward, or produce if requested by any party, because no recording exists.”

Apple states that the Secure Exclave buffer used for initial processing is “a continuously overwritten stream that exists only within the protected hardware and never creates an audio recording,” implying that no one needs to consent to being recorded. In theory, this is the case. However, the fact that Apple felt compelled to explain in such detail how the feature technically avoids violating privacy laws may not make it feel any less dystopian to those who view it that way.

Given the potential security concerns, it is fortunate that Apple has made its Audio Intelligence features opt-in only. They are not enabled by default on your new Apple Watch, and you can choose to enable or disable individual features as you see fit. Here is what to look for:

Sound Recognition: Open the Settings app on your iPhone. Navigate to Accessibility > Sound & Name Recognition > Sound Recognition. Here you can toggle the “Detect using Apple Watch” option on or off.

Music Recognition: Open the Settings app directly on your Apple Watch. Tap on Smart Stack. Switch “Music Detection” on or off per your needs.

Live Rewind: Open the Watch app on your iPhone. Go to Siri > Live Rewind. You can choose to enable or disable “Double Click Digital Crown” to control this feature.

Siri Recap: Again on the Watch app on your iPhone, go to Siri > Siri Recap. You can enable or disable the feature here, or even set an autorun schedule for it if you like. Siri Recap can also be switched on and off by flipping a single toggle in your Control Center.