What is WebNN?

WebNN is a JavaScript API that provides a high-level interface for executing neural network inference tasks efficiently on various hardware accelerators, such as CPUs, GPUs, and dedicated AI chips (sometimes called NPUs or TPUs). By utilizing hardware acceleration, WebNN enables faster and more power-efficient execution of machine learning models, making it ideal for real-time applications and scenarios where latency is critical.

Programming Model

WebNN follows a simple programming model, allowing developers to perform inference tasks with minimal complexity. The API is focused on defining the operations and infrastructure necessary to execute machine learning models, rather than handling higher-level functionalities such as model loading, parsing, or management. WebNN is designed to be agnostic to model formats and leaves the responsibility of loading and parsing models to other libraries (such as ONNX.js or Tensorflow.js) or the web application itself.

At a high level WebNN essentially has 2 steps to run a model:

• Model Construction: In WebNN the first step is to construct the model using the MLGraphBuilder API. Once the model has been constructed it can be built into an executable graph.

• Model Execution: Once the executable graph has been constructed, data is input and the graph executes inference tasks to obtain predictions or classifications. WebNN provides methods for selecting backends (either explicitly or by characteristics) that then process the input data and return output results from the model.

WebNN leverages hardware accelerators to accelerate the execution of models. Since WebNN is hardware and model agnostic it can use any of the available hardware resources (whether CPU, GPU, NPU, TPU, etc), maximizing performance and minimizing latency, enabling smooth and responsive user experiences.

Sample Code

Let’s take a look at example pseudo-code that demonstrates how to perform inference using WebNN. In this example, we will show the basic API for model construction and execution. More details of model construction are covered in other tutorials.

/* Create a context and MLGraphBuilder */
const context = await navigator.ml.createContext(/* execution parameters */);
const builder = new MLGraphBuilder(context);

/* Construct the model */
// WebNN supports a core set of ML operators - a full list can be found at 
// https://www.w3.org/TR/webnn/#api-mlgraphbuilder


/* Build executable graph */
const graph = await builder.build({'output': output});

/* Arrange input and output buffers */
const inputBuffer = new Float32Array(TENSOR_SIZE);
const outputBuffer = new Float32Array(TENSOR_SIZE);

const inputs = {'input': inputBuffer};

const outputs = {'output': outputBuffer};

/* Execute the compiled graph with the specified inputs. */
const results = await context.compute(graph, inputs, outputs);

console.log('Output values: ' + results.outputs.output);

This is a basic pseudo-code example of how to use WebNN API to construct a model and execute it. In a real-world scenario, you would replace the placeholders (/* Construct the model /, / Arrange input and output buffers */, etc.) with actual model construction, input data, and execution parameters specific to your application.

Conclusion

WebNN opens up exciting possibilities for integrating machine learning capabilities into web applications, enabling developers to create innovative experiences powered by AI directly in the browser. With its simple programming model and support for hardware acceleration, WebNN empowers developers to build responsive and efficient AI-driven solutions that run seamlessly on a wide range of devices.

It is a JavaScript implementation of the WebNN API, based on TensorFlow.js that supports multiple backends for both Web browsers and Node.js.

With this polyfill, Web developers are able to experience the WebNN API early before the native implementations are shipped. Meanwhile, it can be treated as an independent implementation to help validate the feasibility and stability of the WebNN specification.

Usage

Import the package via either NPM or a script tag.

• Via NPM

import '@webmachinelearning/webnn-polyfill';

• Via a script tag

<script src="https://cdn.jsdelivr.net/npm/@webmachinelearning/webnn-polyfill/dist/webnn-polyfill.js"></script>

Before using WebNN API, you should set backend to enable TensorFlow.js as follows. Currently WebNN Polyfill supports 3 backends: CPU, WebGL and WASM. CPU backend has higher numerical precision, while WebGL backend provides better performance. A new backend WebGPU is work in progress.

    const backend = 'webgl'; // or 'cpu', 'wasm'
    const context = await navigator.ml.createContext();
    const tf = context.tf;
    await tf.setBackend(backend);
    await tf.ready();

Samples

Try out the live version of WebNN samples which fall back to the WebNN Polyfill in the browser without native WebNN API support. These samples showcase various popular use cases for neural networks powered by WebNN API.

Vision

We will continuously develop the WebNN Polyfill to keep it aligned with the WebNN API spec. At the same time, WebNN API development in Chromium is in full swing, we hope it won’t take too long to bring hardware acceleration access to Web developers through WebNN API.

🚀 Following the two-year incubation period in this Community Group, the W3C has launched the Web Machine Learning Working Group to standardize the Web Neural Network API, now graduating from its incubation stage. This Community Group continues its incubation function for new machine learning capabilities working in parallel with the newly formed Working Group, similarly to e.g. W3C’s WebAssembly and WebGPU efforts.

👏 Huge thanks to all the W3C Community Group and W3C workshop participants for their contributions that have helped shape this work, and W3C for providing a venue to advance this cross-industry effort toward wide adoption.

📢 Please join the Working Group and read more about our journey and what lies ahead of us here or from the W3C blog post.

Anssi Kostiainen
Web Machine Learning Community Group Chair

Machine Learning (ML) is a branch of Artificial Intelligence. A subfield of ML called Deep Learning with its various neural network architectures enables new compelling user experiences for web applications. Use cases range from improved video conferencing to accessibility-improving features, with potential improved privacy over cloud-based solutions. Enabling these use cases and more is the focus of the newly launched Web Machine Learning Working Group.

Progress

While some of these use cases can be implemented in-device in a constrained manner with existing Web APIs (e.g. WebGL graphics API or in the future WebGPU), the lack of access to platform capabilities such as dedicated ML hardware accelerators and native instructions constraint the scope of experiences and leads to inefficient implementations on modern hardware.

With these design goals in mind, a W3C Community Group started incubating work in 2018 for a possible Web Neural Network API, in response to encouraging feedback from a TPAC breakout session. Starting October 2018, this Community Group identified key use cases working with diverse participants including major browser vendors, key ML JS frameworks, interested hardware vendors, and web developers. After identification of the key use cases, the group decomposed the key use cases into requirements and started drafting the Web Neural Network API specification in mid-2019. The aim of this use case-driven design process was to put user needs first.

Quotes

“Having access to the native ML accelerators, machine learning frameworks such as TensorFlow.js can greatly improve model execution efficiency and truly democratize ML for web developers.”

– Ping Yu, TLM for TensorFlow.js at Google

“The early empirical results from the Web Neural Network API implementations demonstrate tremendous power & performance improvements of the Web AI workloads. Through access to the full native AI capabilities of the modern heterogeneous hardware, the Web Neural Network API enables a whole new transformative class of intelligent user experiences on the Open Web Platform across a variety of hardware, software, and device types.”

– Ningxin Hu, Principal Engineer, Web Platform Engineering at Intel

W3C organized a workshop on Web and Machine Learning over the course of August and September 2020. This workshop brought together web platform and machine learning practitioners to survey the broader intersection of Web technologies and Machine Learning, and one of the conclusions of the Workshop was to propose that a new W3C Working Group should be formed to standardize the Web Neural Network API, graduating from its incubation stage. As of 2021, the Community Group continues its incubation function working in parallel with the Working Group, similarly to e.g. W3C’s WebAssembly and WebGPU efforts.

“The Web Neural Network API is a very important step toward the future of the Intelligent Web where AI is infused into the user’s daily web experiences. With the current advances and the pace of innovations in the AI hardware landscape, it’ll help connect those experiences from the clouds and make them personal to the users through seamless native hardware performance on the edge devices across the entire web. That’s the future worth dreaming about!”

– Chai Chaoweeraprasit, Principal Engineering Lead, Machine Learning and Compute Platform at Microsoft

The Web Machine Learning Working Group plans to publish the First Public Working Draft of the Web Neural Network API during the first half of 2021 and welcomes new participants from the diverse W3C community to take part in helping identify new use cases, documenting ethical risks and their mitigations, contributing to technical work, conducting wide reviews in privacy, security, accessibility, and other important areas to ensure the perspectives of the diverse web community are heard. – which given some of the ethical impact of Machine Learning algorithms will be particularly critical to the work of the group. Join us!

We would like to thank all the W3C Community Group and W3C workshop participants for their contributions that have helped shape this work, and W3C for providing a venue to advance this cross-industry effort toward wide adoption.

This post is co-authored by Anssi Kostiainen (Working Group Chair), Ningxin Hu and Chai Chaoweeraprasit (Web Neural Network API Editors), and Ping Yu (TensorFlow.js Core team).

Source: W3C Launches the Web Machine Learning Working Group

It provides several building blocks:

• WebNN C/C++ headers that applications and other building blocks use.
  • The webnn.h that is an one-to-one mapping with the WebNN IDL.
  • A C++ wrapper for the webnn.h
• Backend implementations that use platforms’ ML APIs:
  • DirectML on Windows 10
  • OpenVINO on Windows 10 and Linux
  • oneDNN on Windows 10 and Linux
  • XNNPACK on Windows 10 and Linux
  • Other backends are to be added

WebNN-native uses the code of other open source projects:

• The code generator and infrastructure code of Dawn project.
• The DirectMLX and device wrapper of DirectML project.
• The XNNPACK

Build and Run

Install depot_tools

WebNN-native uses the Chromium build system and dependency management so you need to install depot_tools and add it to the PATH.

Notes:

• On Windows, you’ll need to set the environment variable DEPOT_TOOLS_WIN_TOOLCHAIN=0. This tells depot_tools to use your locally installed version of Visual Studio (by default, depot_tools will try to download a Google-internal version).

Get the code

Get the source code as follows:

# Clone the repo as "webnn-native"
> git clone https://github.com/webmachinelearning/webnn-native.git webnn-native && cd webnn-native

# Bootstrap the gclient configuration
> cp scripts/standalone.gclient .gclient

# Fetch external dependencies and toolchains with gclient
> gclient sync

Setting up the build

Generate build files using gn args out/Debug or gn args out/Release.

A text editor will appear asking build options, the most common option is is_debug=true/false; otherwise gn args out/Release --list shows all the possible options.

To build with DirectML backend, set build option webnn_enable_dml=true.

To build with OpenVINO backend, set build option webnn_enable_openvino=true.

To build with oneDNN backend, set build option webnn_enable_onednn=true.

To build with XNNPACK backend, set build option webnn_enable_xnnpack=true.

Build

Then use ninja -C out/Release or ninja -C out/Debug to build WebNN-native.

Notes

• To build with XNNPACK backend, please build XNNPACK first, e.g. by XNNPACK/scripts/build-local.sh.

Run tests

Run unit tests, for example ./out/Release/webnn_unittests.

Run end2end tests, for example ./out/Release/webnn_end2end_tests.

Notes:

• For OpenVINO backend, please install 2021.2 version and set the environment variables before running the end2end tests.
• For oneDNN backend on Linux, please set the LD_LIBRARY_PATH environment variable to the out folder before running the end2end tests, e.g. LD_LIBRARY_PATH=./out/Release ./out/Release/webnn_end2end_tests.

License

Apache 2.0 Public License.

This example builds, compiles, and executes a graph comprised of three ops, takes four inputs and returns one output.

There are many important application use cases for high-performance neural network inference. One such use case is deep-learning noise suppression (DNS) in web-based video conferencing. The following sample shows how the NSNet2 baseline implementation of deep learning-based noise suppression model may be implemented using the WebNN API.

// Noise Suppression Net 2 (NSNet2)
// Baseline Model for Deep Noise Suppression Challenge (DNS) 2021.
export class NSNet2 {
  constructor() {
    this.context_ = null;
    this.builder_ = null;
    this.graph_ = null;
    this.frameSize = 161;
    this.hiddenSize = 400;
  }

  async load(contextOptions, baseUrl, batchSize, frames) {
    this.context_ = await navigator.ml.createContext(contextOptions);
    this.builder_ = new MLGraphBuilder(this.context_);
    // Create constants by loading pre-trained data from .npy files.
    const weight172 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "172.npy"
    );
    const biasFcIn0 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "fc_in_0_bias.npy"
    );
    const weight192 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "192.npy"
    );
    const recurrentWeight193 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "193.npy"
    );
    const bias194 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "194_0.npy"
    );
    const recurrentBias194 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "194_1.npy"
    );
    const weight212 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "212.npy"
    );
    const recurrentWeight213 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "213.npy"
    );
    const bias214 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "214_0.npy"
    );
    const recurrentBias214 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "214_1.npy"
    );
    const weight215 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "215.npy"
    );
    const biasFcOut0 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "fc_out_0_bias.npy"
    );
    const weight216 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "216.npy"
    );
    const biasFcOut2 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "fc_out_2_bias.npy"
    );
    const weight217 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "217.npy"
    );
    const biasFcOut4 = await buildConstantByNpy(
      this.builder_,
      baseUrl + "fc_out_4_bias.npy"
    );
    // Build up the network.
    const input = this.builder_.input("input", {
      type: "float32",
      dimensions: [batchSize, frames, this.frameSize],
    });
    const relu20 = this.builder_.relu(
      this.builder_.add(this.builder_.matmul(input, weight172), biasFcIn0)
    );
    const transpose31 = this.builder_.transpose(relu20, {
      permutation: [1, 0, 2],
    });
    const initialState92 = this.builder_.input("initialState92", {
      type: "float32",
      dimensions: [1, batchSize, this.hiddenSize],
    });
    const [gru94, gru93] = this.builder_.gru(
      transpose31,
      weight192,
      recurrentWeight193,
      frames,
      this.hiddenSize,
      {
        bias: bias194,
        recurrentBias: recurrentBias194,
        initialHiddenState: initialState92,
        returnSequence: true,
      }
    );
    const squeeze95 = this.builder_.squeeze(gru93, { axes: [1] });
    const initialState155 = this.builder_.input("initialState155", {
      type: "float32",
      dimensions: [1, batchSize, this.hiddenSize],
    });
    const [gru157, gru156] = this.builder_.gru(
      squeeze95,
      weight212,
      recurrentWeight213,
      frames,
      this.hiddenSize,
      {
        bias: bias214,
        recurrentBias: recurrentBias214,
        initialHiddenState: initialState155,
        returnSequence: true,
      }
    );
    const squeeze158 = this.builder_.squeeze(gru156, { axes: [1] });
    const transpose159 = this.builder_.transpose(squeeze158, {
      permutation: [1, 0, 2],
    });
    const relu163 = this.builder_.relu(
      this.builder_.add(
        this.builder_.matmul(transpose159, weight215),
        biasFcOut0
      )
    );
    const relu167 = this.builder_.relu(
      this.builder_.add(this.builder_.matmul(relu163, weight216), biasFcOut2)
    );
    const output = this.builder_.sigmoid(
      this.builder_.add(this.builder_.matmul(relu167, weight217), biasFcOut4)
    );
    return { output, gru94, gru157 };
  }

  async build(outputOperand) {
    this.graph_ = await this.builder_.build(outputOperand);
  }

  async compute(
    inputBuffer,
    initialState92Buffer,
    initialState155Buffer,
    outputBuffer,
    gru94Buffer,
    gru157Buffer
  ) {
    const inputs = {
      input: inputBuffer,
      initialState92: initialState92Buffer,
      initialState155: initialState155Buffer,
    };
    const outputs = {
      output: outputBuffer,
      gru94: gru94Buffer,
      gru157: gru157Buffer,
    };
    const results = await this.context_.compute(this.graph_, inputs, outputs);
    return results.outputs;
  }
}

Try the live version of the WebNN NSNet2 example. This live version builds upon nsnet2.js that implements the above code snippet as a JS module.

The WebNN API brings this abstraction to the web.

In the WebNN API, the Operand objects represent input, output, and constant multi-dimensional arrays known as tensors. The NeuralNetworkContext defines a set of operations that facilitate the construction and execution of this computational graph. Such operations may be accelerated with dedicated hardware such as the GPUs, CPUs with extensions for deep learning, or dedicated ML accelerators. These operations defined by the WebNN API are required by models that address key application use cases.

Additionally, the WebNN API provides affordances to builder a computational graph, compile the graph, execute the graph, and integrate the graph with other Web APIs that provide input data to the graph e.g. media APIs for image or video frames and sensor APIs for sensory data.

This example builds, compiles, and executes a graph comprised of three ops, takes four inputs and returns one output:

const context = navigator.ml.createContext({powerPreference: 'low-power'});

// The following code builds a graph as:
// constant1 ---+
//              +--- Add ---> intermediateOutput1 ---+
// input1    ---+                                    |
//                                                   +--- Mul---> output
// constant2 ---+                                    |
//              +--- Add ---> intermediateOutput2 ---+
// input2    ---+

// Use tensors in 4 dimensions.
const TENSOR_DIMS = [1, 2, 2, 2];
const TENSOR_SIZE = 8;

const builder = new MLGraphBuilder(context);

// Create OperandDescriptor object.
const desc = {type: 'float32', dimensions: TENSOR_DIMS};

// constant1 is a constant operand with the value 0.5.
const constantBuffer1 = new Float32Array(TENSOR_SIZE).fill(0.5);
const constant1 = builder.constant(desc, constantBuffer1);

// input1 is one of the input operands. Its value will be set before execution.
const input1 = builder.input('input1', desc);

// constant2 is another constant operand with the value 0.5.
const constantBuffer2 = new Float32Array(TENSOR_SIZE).fill(0.5);
const constant2 = builder.constant(desc, constantBuffer2);

// input2 is another input operand. Its value will be set before execution.
const input2 = builder.input('input2', desc);

// intermediateOutput1 is the output of the first Add operation.
const intermediateOutput1 = builder.add(constant1, input1);

// intermediateOutput2 is the output of the second Add operation.
const intermediateOutput2 = builder.add(constant2, input2);

// output is the output operand of the Mul operation.
const output = builder.mul(intermediateOutput1, intermediateOutput2);

// Build graph.
const graph = await builder.build({'output': output});

// Setup the input buffers with value 1.
const inputBuffer1 = new Float32Array(TENSOR_SIZE).fill(1);
const inputBuffer2 = new Float32Array(TENSOR_SIZE).fill(1);

// Asynchronously execute the built model with the specified inputs.
const inputs = {
  'input1': {data: inputBuffer1},
  'input2': {data: inputBuffer2},
};
const outputs = await graph.compute(inputs);

// Log the shape and computed result of the output operand.
console.log('Output shape: ' + outputs.output.dimensions);
// Output shape: 1,2,2,2
console.log('Output value: ' + outputs.output.data);
// Output value: 2.25,2.25,2.25,2.25,2.25,2.25,2.25,2.25

Try the live version of the WebNN simple graphs example.

As part of ensuring that the community is aware of proposed work at W3C, this draft charter is public during the Advisory Committee review period.

W3C invites public comments through 03:59 UTC on 2021-03-27 (23:59, Eastern time on 2021-03-26) on the proposed charter. Please send comments to public-new-work@w3.org, which has a public archive lists.w3.org/Archives/Public/public-new-work.

Other than comments sent in formal responses by W3C Advisory Committee Representatives, W3C cannot guarantee a response to comments. If you work for a W3C Member [1], please coordinate your comments with your Advisory Committee Representative. For example, you may wish to make public comments via this list and have your Advisory Committee Representative refer to it from his or her formal review comments.

If you should have any questions or need further information, please contact Dominique Hazael-Massieux, Team Contact for the proposed Web Machine Learning Working Group dom@w3.org.

Source: Proposed W3C Charter: Web Machine Learning Working Group (until 2021-03-26/27)

The mission of the Machine Learning for the Web Community Group (WebML CG) is to make Machine Learning a first-class web citizen by incubating and developing a dedicated low-level Web API for machine learning inference in the browser. Please see the charter for more information.

The group invites browser engine developers, hardware vendors, web application developers, and the broader web community with interest in Machine Learning to participate.

In order to join the group, you will need a W3C account. Please note, however, that W3C Membership is not required to join a Community Group.

This is a community initiative. This group was originally proposed on 2018-10-03 by Anssi Kostiainen. The following people supported its creation: Anssi Kostiainen, Rijubrata Bhaumik, Zoltan Kis, Mike O'Neill, Philip Laszkowicz, Tomoyuki Shimizu. W3C’s hosting of this group does not imply endorsement of the activities.

The group must now choose a chair. Read more about how to get started in a new group and good practice for running a group.

We invite you to share news of this new group in social media and other channels.

If you believe that there is an issue with this group that requires the attention of the W3C staff, please email us at site-comments@w3.org.

Thank you,
W3C Community Development Team