> ## Documentation Index
> Fetch the complete documentation index at: https://docs.streampixel.io/llms.txt
> Use this file to discover all available pages before exploring further.

# How it works

> The parts behind a Streampixel Web SDK stream: your page, the SDK's loader and engine, the platform, and the GPU server that runs your Unreal app. What a session is, how it starts, queues and ends.

Beginner Intermediate For anyone who has a stream on a page and wants to know what happens behind it. After this page you can name each part, say what a session is, and explain every step between `create()` and the first picture.

No code is needed to read this page. If you have not run a stream yet, start with [Your first stream](https://docs.streampixel.io/resources/web-sdk-v2/first-stream.md).

## The parts

Your Unreal app does not run in the browser. It runs on a server with a GPU (Graphics processing unit: the hardware that renders your app's picture.), and the browser shows a live video of it. The viewer's mouse, keyboard and touch input travel the other way.

| Part | Who runs it | What it does |
| - | - | - |
| Your page | You | Holds the element the video goes into, and your own screens and buttons. |
| The Web SDK | Runs in your page | Starts the stream, keeps it alive, and tells your code what happens. It draws only the video. |
| The engine hosts | Streampixel | Serve the SDK's engine: `js.streampixel.io`, with `js-backup.streampixel.io` as a mirror. |
| The platform | Streampixel | Knows your project: whether it is online, how viewers get in. It hands out the ticket for a session. |
| The signalling server | Streampixel | Passes the set-up messages between the browser and your app, so the two can open a video connection. |
| The GPU server | Streampixel | Runs your Unreal app and encodes its picture as video. |

Once the connection is open, video and sound go from your app to the browser, and input and your messages go from the browser to your app. They take a direct path when the network allows one. Otherwise they go through a relay (A server that forwards the video when the viewer's network does not allow a direct path. Also called a TURN relay.).

## The SDK is a loader and an engine

The SDK comes in two pieces. This is the one design choice that affects how you install and update it.

| Piece | Where it is | What it does |
| - | - | - |
| The **loader** | In the npm package, or the `loader.js` script | Small, with a fixed API: `Streampixel.create()` and `Streampixel.preload()`. It finds the engine, checks it, and hands it your options. |
| The **engine** | Downloaded from Streampixel when `create()` runs | Everything that runs the stream: access, signalling, the video connection, input, reconnection, telemetry. |

What this means for you:

* **Fixes reach your site without a new install or deploy.** Streampixel releases a new engine, and your page downloads it the next time it starts a stream (unless you pinned one engine version).
* **The engine is checked before it runs.** The list of engine builds is signed, and the loader verifies the signature with keys built into the package. The engine file must match the hash in that signed list.
* **There is no copy of the engine in the package.** If the engine cannot be downloaded and verified, `create()` rejects with `EngineLoadError` and no stream starts.
* **Two version numbers exist.** `SDK_VERSION` is the package you installed. `stream.engineVersion` is the engine that runs.

> [card] [The engine and its updates](https://docs.streampixel.io/resources/web-sdk-v2/engine-and-updates.md)
>
> Channels, pinning a version, how the engine is verified, and what an update can change.

## What a session is

A **session** is one run of your app for one viewer. Your page starts one with `create()`, and it is over when the state is `ended`.

| Fact | Detail |
| - | - |
| One viewer, one app | By default each session is private: your app runs once for this viewer alone. A [shared session](https://docs.streampixel.io/resources/web-sdk-v2/shared-sessions.md) runs one app for a host and several viewers. |
| A session has ids | `stream.currentSessionId` and `stream.currentStreamerId`. They are the ids shown in your Streampixel analytics. The `session` event announces them. |
| A session needs a ticket | The ticket (A short-lived credential for one streaming session. The platform hands it out after it has checked that the viewer may get in.) is what the platform gives the SDK once the viewer is allowed in. Passwords and single sign-on are checked at this point. |
| A session uses a GPU | For as long as it runs. This is why an idle session is ended, and why viewers can queue. |
| A session can outlive a short network drop | The SDK reconnects on the same session. See [When the connection drops](#when-the-connection-drops). |
| A new session is a new app process | Whatever your app held in memory is gone, and [`launchParams`](https://docs.streampixel.io/resources/web-sdk-v2/options) are sent again. |

## From `create()` to the first picture

This is what happens when your page calls `Streampixel.create()`, with what your code sees at each step:

**1. The engine is downloaded and verified**

Your code sees nothing yet: `create()` has not resolved. If this fails, `create()` rejects with `EngineLoadError`.

**2. The project is looked up**

Steps 4 and 5. The platform says whether the project is online and how viewers get in: public, password, or single sign-on. A project that is switched off or has no active build stops here: `create()` rejects with `AuthError`, kind `project-offline`.

**3. The ticket is requested**

Steps 6 and 7. The answer also carries the project's stream settings from the dashboard: codec, resolution, input, idle time.

A password-protected project pauses before this step. `create()` resolves with the stream in the state `awaitingPassword`, and the ticket is requested when your page calls `submitPassword()`. Nothing has started until then, and no GPU is in use. See [Access](https://docs.streampixel.io/resources/web-sdk-v2/access.md).

**4. create() resolves**

Step 8. Your code has the `stream` object, in the state `idle`. Connecting starts one moment later, so listeners that you attach right after `await Streampixel.create(...)` see everything that follows.

**5. The SDK connects to signalling**

Step 9. The state is `connecting`. Streampixel starts your app on a GPU server. If no GPU is free, the state becomes `queued` (step 10).

**6. Your app and the browser agree on the connection**

Steps 11 to 14. Your app sends an offer, and the browser sends an answer back. The `progress` event fires with `offer`, then `answer`. The offer also means a GPU is assigned: a queued viewer goes back to `connecting`.

**7. The video connection opens**

Step 15. The `progress` event fires with `connected`.

**8. The first picture plays**

Step 16. The state is `streaming`. If the browser refuses to start the video without a click, the state is `awaitingPlay` until the viewer clicks: call `stream.play()` from a button, or let the viewer click the video.

> [!NOTE]
>
> Streams start muted, because browsers want a click before they play sound. Your page turns the sound on with `stream.toggleAudio()` or `stream.setMuted(false)`, from a click. See [Controls and input](https://docs.streampixel.io/resources/web-sdk-v2/controls-and-input.md).

## The states in short

A stream is always in one state. Your page reads it from `stream.state`, and the `state` event fires on every change. The complete list, with every field, is in [Events and states](https://docs.streampixel.io/resources/web-sdk-v2/events-and-states.md). The usual path:

| From | To | When |
| - | - | - |
| (start) | `idle` | `create()` resolves. On a password project it resolves in `awaitingPassword`, and the right password leads on to `connecting`. |
| `idle` | `connecting` | The SDK starts connecting. |
| `connecting` | `queued` | No GPU is free. The state returns to `connecting` when your app answers. |
| `connecting` | `streaming` | The video plays. |
| `streaming` | `awaitingPlay` | The browser wants a click before it plays. The viewer's click leads back to `streaming`. |
| `streaming` | `reconnecting` | The connection drops. It returns to `streaming` when restored, or goes to `ended` when it is not restored in time. |
| `streaming` | `ended` | The session is over. |

One more state exists for recovery: `recovering`. The state is also `connecting` again while a connection is rebuilt. Any state can lead to `ended`.

## The queue

A session needs a GPU. When none is free, the viewer waits.

* The state is `queued`, with a `position`. The `queue` event fires when the position changes.
* `position` is a whole number. It is `0` when the server did not say.
* The viewer leaves the queue when your app answers: the state goes back to `connecting`, then `streaming`.
* Your page only has to show it. With [`@streampixel/ui`](https://docs.streampixel.io/resources/web-sdk-v2/ui-overlays) the loading screen shows the position. For your own display, see the [Cookbook](https://docs.streampixel.io/resources/web-sdk-v2/cookbook.md).

## When the connection drops

A short network problem does not end a session. The SDK handles it by itself, and your page only shows the state.

| What happens | What the SDK does | State |
| - | - | - |
| The connection drops | Reconnects to the same session, for up to 180 seconds in a normal session. | `reconnecting`, then `streaming` |
| No video arrives | Starts one fresh session: a new app process, so what your app held in memory is lost. | `recovering`, then `connecting` and `streaming` |
| Neither works | Ends the session with a code that says why. | `ended` |

The exact timings and every case are in [Reliability](https://docs.streampixel.io/resources/web-sdk-v2/reliability.md). What your page sees is in [Reconnection and recovery](https://docs.streampixel.io/resources/web-sdk-v2/events-and-states.md#reconnection-and-recovery).

## What ends a session

Every end has a code, a kind (`endKind`) and a sentence for the viewer (`message`). Your page gets them in the `ended` event and in the `ended` state.

| Who ends it | For example | `endKind` |
| - | - | - |
| Your code | You call `stream.disconnect()` or `stream.destroy()`, or abort the `signal` you passed to `create()`. | `user` |
| An idle viewer | No input for the idle time, and nobody answers the countdown. The idle time is your project's setting, or 600 seconds. On phones and tablets: the tab was in the background for 60 seconds. | `afk` |
| The platform | The project was switched off. The session reached its maximum runtime. Your app closed unexpectedly. No streaming server was available. | `server` |
| The SDK gives up | A dropped connection was not restored in time. No video arrived, also after one fresh session. | `error` |
| The viewer's network | It allows neither a direct path nor the relay. | `network-blocked` |

> [!WARNING]
>
> `ended` is final. A stream that has ended cannot be restarted, and all its listeners are removed after the `ended` event. To start again, call `stream.destroy()` and then `Streampixel.create()` again.

All codes, with the sentence for each: [How a session ends](https://docs.streampixel.io/resources/web-sdk-v2/errors-and-disconnects.md#how-a-session-ends).

## Glossary

| Term | Meaning |
| - | - |
| **Project** | Your Unreal app on Streampixel, with its settings in the dashboard. |
| **Project ID** | The identifier of a project. The SDK option is named `appId`. |
| **Container** | The HTML element your page gives to the SDK. The SDK puts the video in it. |
| **Loader** | The small part of the SDK that you install or load with a script tag. |
| **Engine** | The part of the SDK that runs the stream. The loader downloads it when `create()` runs. |
| **Platform** | Streampixel's service that knows your project and hands out tickets. |
| **Ticket** | A short-lived credential for one streaming session. |
| **Session** | One run of your app for one viewer. |
| **Signalling** | The connection (a WebSocket) on which the browser and your app agree on the video connection. |
| **WebRTC** | The browser's built-in technology for live video, sound and data. The SDK uses it to show your app. |
| **Offer and answer** | The two set-up messages of a WebRTC connection. Your app sends the offer, the browser sends the answer. |
| **Relay (TURN)** | A server that forwards the video when the network does not allow a direct path. |
| **Queue** | Where a viewer waits when no GPU is free. |
| **State** | What the stream is doing now: `connecting`, `queued`, `streaming`, `ended` and others. |
| **Idle time (AFK)** | How long a viewer may give no input before the countdown starts. |
| **Share page** | Streampixel's own hosted page for a project, the one behind its share link. |
| **Dashboard** | Where you manage your projects: [dashboard.streampixel.io](https://dashboard.streampixel.io). |

## Next steps

> [card] [Installation](https://docs.streampixel.io/resources/web-sdk-v2/installation.md)
>
> npm or a script tag, TypeScript, frameworks, Content-Security-Policy.

> [card] [Quick start](https://docs.streampixel.io/resources/web-sdk-v2/quick-start.md)
>
> A real page with npm and the ready-made screens.

> [card] [Events and states](https://docs.streampixel.io/resources/web-sdk-v2/events-and-states.md)
>
> Every state and every event, with its fields.

> [card] [Under the hood: reliability](https://docs.streampixel.io/resources/web-sdk-v2/reliability.md)
>
> What the SDK does when things go wrong, with the real timings.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.