---
title: "How it works"
description: "The signal pipeline from raw audio to rendered bars — and where the FFT actually runs."
canonical_url: "https://www.fftvisualizer.com/guide/how-it-works"
---
# How it works

> The signal pipeline from raw audio to rendered bars — and where the FFT actually runs.

FFT Visualizer is, at its core, a **renderer**: give it an array of magnitudes and it draws
a spectrum on the GPU. Everything before that — capturing audio and turning it into
frequency data — is *pluggable*, and **where the FFT runs depends on the mode** you pick.

## The pipeline

```text
mode="local"     mic / tab audio
                        │  getUserMedia() / getDisplayMedia()
                        ▼
                  Web Audio:  AudioContext ▸ MediaStreamSource ▸ AnalyserNode
                        │  getFloatTimeDomainData()  ← raw waveform (e.g. 2048 samples)
                        ▼
                  Rust / WASM  FftProcessor.process()
                    Hann window ▸ real FFT ▸ magnitude
                    ▸ log-spaced bands ▸ A-weighting
                        │
                        ▼
                  ┌───────────────────────────────┐
 mode="websocket" │  Uint8Array — 0–255,          │  ◀── a backend server streams this
 mode="external"  │  one value per frequency band │  ◀── you push this via props
                  └───────────────────────────────┘
                        │
                        ▼  noise floor ▸ smoothing ▸ peak-hold ▸ aggregate to `bands`
                        ▼
                  WebGL fragment shader — one draw call
                        │
                        ▼
                      bars on screen
```

The `Uint8Array` of 0–255 magnitudes is the **hand-off point**. In `local` mode the
visualizer produces it; in `websocket` and `external` modes it arrives ready-made and the
FFT is **not** part of the visualizer at all.

## Where the FFT runs

<table>
<thead>
  <tr>
    <th>
      Mode
    </th>
    
    <th>
      Who computes the FFT
    </th>
    
    <th>
      Runs where
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        local
      </code>
    </td>
    
    <td>
      The bundled Rust/WASM processor
    </td>
    
    <td>
      In the browser
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        websocket
      </code>
    </td>
    
    <td>
      Your backend (Python / Node / Rust examples)
    </td>
    
    <td>
      On a server or device
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        external
      </code>
    </td>
    
    <td>
      Your own code (Web Audio, another analyser…)
    </td>
    
    <td>
      Wherever you like
    </td>
  </tr>
</tbody>
</table>

So if you're wondering "is the audio→FFT conversion part of FFTVisualizer?" — only in
`local` mode. See [Data modes](https://www.fftvisualizer.com/guide/data-modes) for the API of each.

## Inside `local` mode

When you set `mode` to `local`, the visualizer runs the whole chain client-side:

1. **Capture** — `getUserMedia()` (microphone) or `getDisplayMedia()` (tab/system audio)
returns a `MediaStream`.
2. **Web Audio graph** — `AudioContext → MediaStreamAudioSourceNode → AnalyserNode`.
3. **Grab raw samples** — each animation frame it reads
`analyser.getFloatTimeDomainData()`, which returns the **raw waveform** (the audio
samples themselves), *not* frequency data.
4. **FFT in Rust/WASM** — those samples go to `FftProcessor.process()`, which:

  - applies a **Hann window** (reduces spectral leakage),
  - runs a **real FFT**,
  - takes the **magnitude** of each complex bin (normalised by FFT size),
  - groups bins into **exponentially (log-)spaced frequency bands**, so bass and treble
  get perceptually fair spacing instead of linear,
  - applies **A-weighting** (the loudness curve that matches how the ear weights
  frequencies).
5. The result is a `Uint8Array` of 0–255 values — one per band — ready for the shader.

The WASM module is **lazy-loaded**: it's only fetched the first time you actually capture
audio locally, so `websocket`/`external` users never download it.

<note title="Why not use the browser's built-in FFT?">

`AnalyserNode` can compute the FFT for you (`getByteFrequencyData`). FFT Visualizer
deliberately grabs raw **time-domain** samples and runs its **own** FFT instead, for two
reasons: **identical output everywhere** — the exact same Rust code runs in the browser
(WASM) *and* in the [backend reference servers](https://www.fftvisualizer.com/guide/websocket-protocol#reference-servers),
so a local mic demo and a remote stream look the same — and **control** the browser doesn't
expose: custom windowing, log-spaced bands, A-weighting and a fixed frequency range
(100 Hz–18 kHz). The `AnalyserNode` here is used purely as a convenient way to read a buffer
of samples; its own FFT is ignored.

</note>

## Two resolutions: `bins` vs `bands`

There are two spectrum sizes in the chain, and it's worth keeping them straight:

- **bins** — the number of frequency bands the FFT processor (or your server) emits.
- **bands** — how many bars the visualizer actually *displays*. It aggregates the incoming
`bins` down to `bands` before rendering.

They're often the same (e.g. 80 → 80), but you can render fewer bars than you receive — for
example feed 80-bin data but display `bands: 28`.

<tip>

Every mode converges on the **same** 0–255 array before rendering — so the look you
configure is driven by identical data whether it came from a local mic, a remote server, or
your own pipeline. Want that array back out — to drive LEDs, a flip-dot display, or any
external hardware? The `frame` event ([Core](https://www.fftvisualizer.com/core/api#events), [Vue](https://www.fftvisualizer.com/vue/props#the-frame-event),
[React](https://www.fftvisualizer.com/react/props#onframe)) hands you exactly what the shader draws, every frame.

</tip>


## Sitemap

See the full [sitemap](https://www.fftvisualizer.com/sitemap.md) for all pages.