Skip to main content
Intermediate For web developers who build their own buttons around a stream. After this page you can add a sound button, a resolution picker, fullscreen, text input for phones, and microphone and camera switches.
You do not have to build any of this. @streampixel/ui draws a control bar with sound, stream information and fullscreen, the “Click to start” screen and the text field for phones. Read on when you want your own.
Every setting on this page has the same three layers: For the settings that cost GPU time or bandwidth, the dashboard is the upper limit: your code can ask for less, never more. The full list is in Options.

Sound and autoplay

A stream starts muted. Browsers only allow sound after the viewer has clicked or tapped. Give the viewer a sound button.
1

Add a button

2

Switch the sound in its click handler

Sound button
The video plays without sound. After one click on the button you hear the application.
boolean
Whether the sound is muted now. true before the video exists.
boolean
Mutes or unmutes. Returns the muted state now in effect. Call it from a click or tap when you unmute.
boolean
Flips mute. Returns true when the sound is now on. Note the direction: it is the opposite of muted.
Read stream.muted each time the state becomes streaming, as the example does. When the SDK rebuilds a connection it creates a new video element, and a new video element starts muted.

When the browser blocks autoplay

Some browsers refuse to start even a muted video without a click. The stream is then up, and nothing plays. Show a “Click to start” control that calls stream.play():
Click to start
When you passed a container, a press anywhere on it also starts playback. The state becomes streaming when the video really plays.

Resolution and quality

How the video fits its container

The resolution mode decides the size of the stream and how it sits in your container. It is the project’s dashboard setting, or the resolution.mode option.
For dynamic and crop, give the container a width and a height. A container with no size is measured as the whole browser window.

Change the resolution

1

List the sizes the project allows

stream.allowedResolutions holds the sizes up to the project’s maximum, for example "720p (1280x720)".
2

Ask for one

stream.setResolution(width, height) asks the application to render at that size.
3

Read what was applied

It returns the size that was sent, or null when nothing was sent.
Resolution picker
What setResolution() does with your numbers:

Set quality at the start

These options apply from the first frame. Each defaults to the project’s dashboard setting.
Quality options
The SDK orders the codecs that the viewer’s browser can decode. A browser without the primary codec uses the fallback, then its own default. AV1 is used only when the application was built with Unreal Engine above 5.3.
bitrate is in kilobits per second. 20_000 is 20 Mbps. A value above 200,000 logs a console warning, because it looks like bits per second.
To show the viewer what they are getting, read the stats event: frame rate, bitrate, resolution, codec. See StreamStats and the stats readout recipe.

Fullscreen

The SDK has no fullscreen method. Use the browser’s own Fullscreen API on the element that holds the stream.
index.html
Fullscreen button
  • Make the wrapper fullscreen, not the stream container: your own buttons inside the wrapper then stay visible.
  • In dynamic and crop modes the stream follows the container, so it is rendered at the new size after a moment. In fixed mode the picture is scaled.

Mouse, keyboard, touch and gamepad

Input goes to your application as soon as the video is up. Each kind follows the project’s dashboard setting (Input and UI controls) unless you pass an option.
Input options
boolean
default:"dashboard, else on"
Mouse movement, buttons and wheel.
boolean
default:"dashboard, else on"
Keys.
boolean
default:"dashboard, else on"
Touches on phones, tablets and touch screens.
boolean
default:"dashboard, else off"
Sends touches as mouse events. For an application that has no touch handling of its own.
boolean
default:"dashboard, else off"
Gamepads connected to the viewer’s device.
boolean
default:"dashboard, else off"
Hover mouse. See below.
boolean
default:"dashboard, else off"
WebXR controllers. See VR and XR.
A viewer of a shared session watches only. Mouse, keyboard, touch, gamepad, microphone and camera are off for a viewer, whatever the options say. The host has the input.

Hover mouse

With hover mouse on, mouse movement reaches the application without a pressed button. Whether your application wants that depends on how it shows its cursor: see Mouse settings. Three things can switch it:
A change made while streaming belongs to the current connection. When the SDK rebuilds the connection, hover mouse starts again from the option or the dashboard. If your page changed it, set it again when the state returns to streaming.

VR and XR

The Unreal side is on Prepare your Unreal Engine project for VR.

Text input on phones

A phone has no keyboard until a text field on the page has focus. When a text field in your application gets focus, the application asks the page for one. The Unreal side is on Virtual keyboard for text input.
1

Add a text field to your page

2

Show it on osk, answer with sendTextboxEntry()

Text input for the application
Tap a text field in the stream on a phone. Your field appears with the same text. Type, press Send, and the text is in the application’s field.
  • Keys typed into your field are not sent to the application as key presses. Only sendTextboxEntry() sends the text.
  • Some phones open the keyboard only when the viewer taps your field. Place the field where it is easy to tap.
  • The event fires on every device, not only on phones.

Microphone and camera

The viewer’s microphone or camera can be sent to your application, for example for voice commands. This is not voice chat between people: for that, see Voice and text chat.
1

Allow it at the start

Switch it on in the dashboard, or pass media. This must be decided before the stream starts.
Allow the microphone
With the microphone allowed, the SDK asks the browser for it as soon as the video is up, and sends it once the viewer agrees. A prompt that the viewer ignores never delays the video.
2

Switch it while streaming

Microphone button
The SDK does not report the viewer’s answer to the first permission prompt. setMicrophone(true) does: it resolves to true only when the microphone is live.
Promise<boolean>
Turns the viewer’s microphone on or off while the stream is up. Resolves to whether it is live now. Call it from a click or tap: the first call asks the browser for permission.
Promise<boolean>
The same for the camera (media.camera).
Both resolve to false in these cases:
The microphone is sent as it is: one channel, with the browser’s echo cancellation, noise suppression and automatic gain switched off.

Idle viewers and background tabs

Input also tells the platform that the viewer is still there. The recipe for your own warning is in the cookbook.

Troubleshooting

Streams start muted. Call stream.setMuted(false) or stream.toggleAudio() from a click or tap. Outside a click the browser may refuse.
The SDK built a new connection, with a new video element that starts muted. Read stream.muted when the state becomes streaming and update your button.
The browser blocked autoplay. The state is awaitingPlay. Show a button that calls stream.play().
The stream is not up yet, a value is under 64, or this is a viewer of a shared session.
In dynamic and crop modes the stream is sized to the container. Check that the container has a width and a height. Without them the SDK measures the browser window.
Keys typed in <input>, <textarea>, <select> and contenteditable elements are held back. A custom widget built from other elements is not recognised.
The microphone was not allowed at the start. Pass media: { mic: true } to create() or switch it on in the dashboard, then start a new stream.
Before the viewer’s first mouse press the SDK sends one middle-button press and release at position 0,0. If your application reacts to the middle button, ignore a press at that position.

Next steps

Talk to your Unreal app

Messages, console commands, files and screenshots.

Events and states

One level up: every state and event behind these controls.

Options reference

Every option of create(), with types and defaults.

Stream API reference

Every method and property of the stream object.