This page explains how the SDK handles stream audio playback and how to forward the user’s microphone and camera to Unreal Engine.
Stream audio playback
The SDK creates two separate media elements for the stream:
- A video element for the video feed
- A separate audio element for the audio feed
Both elements must be managed when muting/unmuting the stream audio.
The SDK uses separate video and audio elements. When muting or unmuting, you must control both elements — toggling just the video element will not affect the audio stream, and vice versa.
Toggling audio with UIControl
The simplest way to toggle stream audio:
This toggles the audio element’s muted state.
Manual audio control
For more control, access both elements directly:
// Get references to both elements
const video = appStream.stream.videoElementParent.querySelector('video');
const audio = appStream.stream._webRtcController.streamController.audioElement;
// Mute both
video.muted = true;
audio.muted = true;
// Unmute both
video.muted = false;
audio.muted = false;
// The audio element may be paused — resume it
if (audio.paused) {
audio.play().catch(() => {});
}
Browser autoplay policy
Browsers require a user gesture before playing audio. The SDK starts with video muted (StartVideoMuted: true) and audio auto-playing silently. You should unmute in response to a user action (e.g., clicking an “Unmute” button).
Browsers block audio playback until the user interacts with the page (click, tap, or keypress). Always unmute audio inside a user-triggered event handler — calling audio.play() without a gesture will be rejected by the browser.
Configuration
await StreamPixelApplication({
appId: 'your-project-id',
useMic: true,
});
Enabling at runtime
// Request microphone permission and enable
try {
await navigator.mediaDevices.getUserMedia({ audio: true });
pixelStreaming.unmuteMicrophone(true);
} catch (err) {
console.error('Microphone access denied:', err.message);
}
Configuration
await StreamPixelApplication({
appId: 'your-project-id',
useCamera: true,
});
Enabling at runtime
// Request camera permission and enable
try {
await navigator.mediaDevices.getUserMedia({ video: true });
pixelStreaming.unmuteCamera(true);
} catch (err) {
console.error('Camera access denied:', err.message);
}
Both microphone and camera require the user to grant browser permissions. Always wrap getUserMedia calls in try/catch to handle denied permissions gracefully.
Next steps
Input controls
Configure mouse, keyboard, touch, gamepad, and WebXR forwarding.
Video, codecs, and resolution
Tune codec selection, resolution presets, and bitrate.
