On-device AI in the browser: how it works, and what it can and cannot do

Why some AI tools run on your own computer, what happens during the model download and how to judge the results.

Most people picture artificial intelligence as a service in a distant data centre: you send your text or photo, and the answer comes back. A growing number of AI tasks can now run inside an ordinary web browser, on your own computer or phone. That changes the privacy picture, but also brings limits that are worth understanding.

What "running in the browser" means

An AI model is essentially a large file of numbers, called weights, together with a set of mathematical operations. To run on your device, the browser downloads the model and performs those operations locally using WebAssembly on the processor, or, where available, the graphics chip through WebGPU.

The inputs you give the tool are then processed on your machine. For an image or a document, there is no need to upload the content to a server for the actual AI inference. That differs from a typical cloud service, where your input must leave your device.

Why the first run is slower

The first time you use such a tool, the browser must fetch the model. Sizes vary from a few megabytes for simple tasks to hundreds of megabytes or more for language models. After that, the browser usually caches it so the next visit starts much faster. If you clear your site data or the browser removes stored files to free up space, the model will be downloaded again.

On a mobile connection, check the size before you begin, since it may count against your data plan. It is also normal for the first prediction to take longer, as the model must be loaded into memory and prepared.

What affects the speed

Performance depends heavily on your hardware:

  • A device with a recent graphics chip and WebGPU support can be many times faster than the processor alone.
  • Available memory matters. Large models can fail on phones or on computers with little free RAM.
  • Longer text and higher-resolution images need more time.
  • Thin and light devices may slow down when they get hot during sustained work.

If a tool is slow or crashes, close other tabs, try a smaller input and use an up-to-date browser. Heavy models simply may not be practical on an older phone.

The privacy benefits and their boundaries

On-device processing is a real privacy advantage for personal photos, private documents or confidential notes, since the content does not have to be sent to anyone for inference. It is good to stay realistic about what that means, though:

  • The page itself, and the model files, are still downloaded from the internet.
  • A website can still use ordinary analytics and advertising, as described in its privacy policy.
  • Browser extensions may have access to what you see on a page.
  • Whatever you later do with the result, such as e-mailing it, is a separate matter.

For especially sensitive material, consider following your organisation's policy, even with a tool that runs locally.

Why smaller models make different mistakes

Models that fit in a browser are normally smaller than those in large data centres. They often specialise in one narrow task, and they can perform very well within it: removing a background, recognising printed text or classifying sounds. For open-ended language tasks, a compact model will be noticeably weaker than a large cloud model, particularly in languages that have less training data, Turkish among them.

Common failure patterns include:

  • Summaries that leave out key points or state something that was not in the source.
  • Translations that sound fluent but change the meaning, names or numbers.
  • OCR that confuses similar characters or struggles with handwriting.
  • Detail "invented" by image enhancers, such as faces or text that look plausible but are not accurate.

Checking the output

Treat results as a draft that needs review, in proportion to what depends on them.

  • Compare names, dates, amounts and quotations with the source.
  • Have a person check translations of legal, medical or official material.
  • Do not use enhanced images as evidence of what a scene really contained.
  • Keep the original files, since AI-generated changes may not be reversible.

Confident wording does not indicate correctness. A model can produce a fluent and entirely wrong sentence without any signal that something is off.

Using AI fairly

Use these tools on content you own or have permission to process. Be careful about altering images of real people, and do not present synthetic or heavily edited pictures as genuine. Removing copyright notices or watermarks from someone else's work is not an acceptable use, whatever the technology makes possible. If AI played a substantial part in something you publish, being open about that helps your readers judge it.

When a cloud service is the better choice

Local tools are not always the right fit. A large cloud model may be a better option for long reasoning tasks, for high-quality writing, or when your device cannot run a local one, provided that you are comfortable with its data policy. A good rule of thumb is to keep private material and simple, well-defined tasks local, and to reserve cloud services for complex work that does not involve sensitive information.

Tools mentioned in this guide