feat: detect speech natively for local audio tracks

docs/guide/audio/audio-processing.md

@@ -296,6 +296,23 @@ audioProcessing.processStream(
     highRateBuffer, highRateConfig, lowRateConfig, lowRateDestBuffer);
 ```
 
+## Speech in the Processed Microphone Audio
+
+WebRTC passes the microphone's audio from the audio device module straight to the senders, so a sink on a microphone track gets nothing. The factory detects speech in that audio as it is sent, after echo cancellation, noise suppression and gain control, in native code, and tells a listener only when speech begins and ends:
+
+```java
+VoiceActivityListener listener = speaking -> {
+    // On WebRTC's capture thread: hand it on and return.
+};
+
+factory.addVoiceActivityListener(new VoiceActivityConfig(0.5, Duration.ofMillis(300)), listener);
+
+// Later, to stop it.
+factory.removeVoiceActivityListener(listener);
+```
+
+It works with the `AudioProcessing` the factory was built with as well as with the one it builds itself. All listeners share one detector, and each decides with the threshold and hangover of its own `VoiceActivityConfig`. Audio arrives while the audio device module records, which WebRTC starts once a peer connection sends a microphone track. A listener must not add or remove a listener. Disposing of the factory removes the listeners it still has. The [Client API](/guide/client-api#audio) offers the same through `LocalAudio.onVoiceActivity`.
+
 ## Statistics
 
 The `AudioProcessing` class provides statistics about the audio processing performance:

docs/guide/client-api.md

@@ -522,6 +522,17 @@ A handler that falls behind gets the audio of the last 5 seconds; the oldest aud
 track.asAudio().onAudio(Duration.ofMillis(200), chunk -> recognizer.feed(chunk.samples()));
 ```
 
+A local audio track tells when this side speaks. `onVoiceActivity` calls its handler with `true` when speech begins and `false` when it ends, on the event executor. The audio is analyzed in native code, and only these changes reach Java:
+
+```java
+LocalAudio microphone = rtc.media().microphone();
+Subscription speaking = microphone.onVoiceActivity(
+        new VoiceActivityConfig(0.6, Duration.ofMillis(400)),
+        active -> System.out.println(active ? "speaking" : "quiet"));
+```
+
+`VoiceActivityConfig` sets the threshold, the voice probability from 0 to 1 at which audio counts as speech (0.5 by default), and the hangover, how long speech goes on once the audio fell below it (300 ms by default), which bridges the pauses between words. Pushed audio is analyzed as pushed. A microphone's audio is analyzed after audio processing, with echo cancellation, noise suppression and gain control applied as configured, so the remote peer's voice from the speakers does not count as this side speaking. It flows while WebRTC records, which it does while a peer sends a microphone track. Every microphone track of a runtime hears the same, as WebRTC records one microphone for all of them.
+
 ### Receiving Without Sending
 
 A viewer that sends nothing asks for what it wants to receive, so that the offer includes it:

docs/tools/audio/voice-activity-detector.md

@@ -13,6 +13,10 @@ The existing `VoiceActivityDetector.process()` and `getLastVoiceProbability()`
 methods remain available with the original probability estimator. No migration
 is required. The standalone detector is an additional class, with its own state.
 
+To learn only when speech begins and ends in a stream, without handing its
+audio to Java, add a `VoiceActivityListener`, see
+[Speech events in native code](#speech-events-in-native-code).
+
 ## Probability estimator
 
 `dev.onvoid.webrtc.media.audio.VoiceActivityDetector` combines the standalone VAD
@@ -118,6 +122,50 @@ they receive the same audio. Feeding, configuring, resetting, or disposing one
 does not update the other. Neither API automatically changes encoding, DTX, mute
 state, or transmitted audio.
 
+## Speech events in native code
+
+A `VoiceActivityListener` gets the changes only: `true` when speech begins and
+`false` when it ends. The probability estimator runs in native code on the
+audio as it flows, and a `VoiceActivityConfig` sets when audio counts as speech:
+
+- the threshold, the probability from which on audio counts as speech, above 0
+  and at most 1, 0.5 by default;
+- the hangover, how long speech goes on once the audio fell below the
+  threshold, 300 ms by default, which bridges the pauses between words.
+
+Audio of any rate and channel count is mixed down to mono and resampled to
+16 kHz first. The estimator's initial probability of `1.0` never counts.
+
+An `AudioTrack` whose source hands its audio to sinks, a remote track or a
+track over a `CustomAudioSource`, takes a listener for its audio:
+
+```java
+VoiceActivityListener listener = speaking -> {
+    // On the thread that delivers the audio: enqueue it and return promptly.
+};
+track.addVoiceActivityListener(new VoiceActivityConfig(), listener);
+try {
+    // Keep the listener while receiving audio.
+}
+finally {
+    track.removeVoiceActivityListener(listener);
+}
+```
+
+A microphone track does not hand its audio to sinks. Its speech comes from the
+factory, after echo cancellation and noise suppression, so the remote peer's
+voice from the speakers does not count as this side speaking:
+`PeerConnectionFactory.addVoiceActivityListener`, see
+[Audio Processing](/guide/audio/audio-processing#speech-in-the-processed-microphone-audio).
+With the [Client API](/guide/client-api#audio), `LocalAudio.onVoiceActivity`
+covers both, for the microphone and for pushed audio:
+
+```java
+Subscription speaking = microphone.onVoiceActivity(active -> {
+    // On the event executor.
+});
+```
+
 ## Audio track integration
 
 The examples module includes `VoiceActivityExample`, an adapter using
@@ -167,8 +215,10 @@ finally {
 
 A microphone track does not: WebRTC passes captured audio from the audio device
 module straight to the senders, and the source of the track hands nothing to
-sinks. An [`AudioRecorder`](/tools/audio/audio-recorder) captures a microphone
-on its own, apart from any peer connection.
+sinks. Its speech comes from a listener on the factory, see
+[Speech events in native code](#speech-events-in-native-code). An
+[`AudioRecorder`](/tools/audio/audio-recorder) captures a microphone on its
+own, apart from any peer connection.
 
 The adapter invokes the consumer on the thread that delivers the audio: a thread
 of the `onAudio` subscription, or the audio callback thread of a sink. Keep work

webrtc-jni/src/main/cpp/include/media/audio/AudioCaptureTap.h

@@ -0,0 +1,72 @@
+/*
+ * Copyright 2026 Alex Andres
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef JNI_WEBRTC_MEDIA_AUDIO_CAPTURE_TAP_H_
+#define JNI_WEBRTC_MEDIA_AUDIO_CAPTURE_TAP_H_
+
+#include "media/audio/VoiceActivityListener.h"
+#include "media/audio/VoiceDetector.h"
+
+#include "api/audio/audio_processing.h"
+
+#include <cstdint>
+#include <memory>
+#include <mutex>
+#include <vector>
+
+namespace jni
+{
+	// Detects speech in the capture signal of an AudioProcessing, after all
+	// of its processing: what the senders send. The module calls it through
+	// its capture post-processor, which is put in when the module is built,
+	// so the tap does nothing until a listener is added. All listeners share
+	// one detector; each decides with its own threshold and hangover.
+	//
+	// The tap is shared: by the post-processor, for as long as the module
+	// lives, and by the Java objects that hold it, the module's and the
+	// factory's, for as long as they live. None of them outlives it.
+	class AudioCaptureTap
+	{
+		public:
+			// The capture post-processor to build an AudioProcessing with.
+			static std::unique_ptr<webrtc::CustomProcessing> CreateProcessing(std::shared_ptr<AudioCaptureTap> tap);
+
+			// Adds a listener, which hears of the speech from the next 10 ms
+			// on. Returns it, as the handle that removes it.
+			VoiceActivityListener * AddListener(std::unique_ptr<VoiceActivityListener> listener);
+
+			// Removes a listener added and deletes it. Once this returns, it is
+			// no longer called. Must not be called from within a listener.
+			void RemoveListener(VoiceActivityListener * listener);
+
+			// Called on the capture thread.
+			void Process(const webrtc::AudioBuffer & audio);
+
+		private:
+			std::mutex mutex;
+
+			// Guarded by mutex.
+			std::vector<std::unique_ptr<VoiceActivityListener>> listeners;
+
+			// Guarded by mutex.
+			VoiceDetector detector;
+
+			// Guarded by mutex.
+			std::vector<int16_t> samples;
+	};
+}
+
+#endif

webrtc-jni/src/main/cpp/include/media/audio/VoiceActivityListener.h

@@ -0,0 +1,69 @@
+/*
+ * Copyright 2026 Alex Andres
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef JNI_WEBRTC_MEDIA_AUDIO_VOICE_ACTIVITY_LISTENER_H_
+#define JNI_WEBRTC_MEDIA_AUDIO_VOICE_ACTIVITY_LISTENER_H_
+
+#include "JavaClass.h"
+#include "JavaRef.h"
+
+#include <jni.h>
+#include <memory>
+
+namespace jni
+{
+	// Decides from the voice probability of each 10 ms chunk when speech
+	// begins and ends, and tells a Java VoiceActivityListener of each change,
+	// and of nothing else. Speech begins with a chunk at or above the
+	// threshold, and ends once the chunks stayed below it for longer than the
+	// hangover.
+	class VoiceActivityListener
+	{
+		public:
+			VoiceActivityListener(JNIEnv * env, const JavaGlobalRef<jobject> & listener, float threshold,
+				int hangoverMs);
+			~VoiceActivityListener() = default;
+
+			// Called on the thread that delivers the audio.
+			void Update(float probability);
+
+		private:
+			class JavaVoiceActivityListenerClass : public JavaClass
+			{
+				public:
+					explicit JavaVoiceActivityListenerClass(JNIEnv * env);
+
+					jmethodID onVoiceActivity;
+			};
+
+		private:
+			JavaGlobalRef<jobject> listener;
+
+			const std::shared_ptr<JavaVoiceActivityListenerClass> javaClass;
+
+			const float threshold;
+
+			// The hangover in chunks, rounded up.
+			const int hangoverChunks;
+
+			// The chunks below the threshold since speech was last detected.
+			int quietChunks;
+
+			bool speaking;
+	};
+}
+
+#endif

webrtc-jni/src/main/cpp/include/media/audio/VoiceActivitySink.h

@@ -0,0 +1,49 @@
+/*
+ * Copyright 2026 Alex Andres
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef JNI_WEBRTC_MEDIA_AUDIO_VOICE_ACTIVITY_SINK_H_
+#define JNI_WEBRTC_MEDIA_AUDIO_VOICE_ACTIVITY_SINK_H_
+
+#include "media/audio/VoiceActivityListener.h"
+#include "media/audio/VoiceDetector.h"
+
+#include "api/media_stream_interface.h"
+
+#include <memory>
+
+namespace jni
+{
+	// A sink of an audio track that detects speech in what the source of the
+	// track hands to sinks, and tells a listener when it begins and ends.
+	class VoiceActivitySink : public webrtc::AudioTrackSinkInterface
+	{
+		public:
+			explicit VoiceActivitySink(std::unique_ptr<VoiceActivityListener> listener);
+			~VoiceActivitySink() = default;
+
+			// AudioTrackSinkInterface implementation. The other variants
+			// forward to this one.
+			using webrtc::AudioTrackSinkInterface::OnData;
+			void OnData(const void * data, int bitsPerSample, int sampleRate, size_t channels, size_t frames) override;
+
+		private:
+			VoiceDetector detector;
+
+			const std::unique_ptr<VoiceActivityListener> listener;
+	};
+}
+
+#endif