O'Reilly logo
live online training icon Live Online training

Getting Started with TensorFlow.js

Machine Learning in the Browser

Brian Sletten

Several trends in our industry are colliding and creating new opportunities while complicating our technology choices. The Web continues to be the major platform for deploying modern applications in lightweight, zero-installation and cross-platform environments. Machine Learning is emerging as a way of managing the explosion of data that we no longer have the capacity to approach with conventional strategies. Hardware is increasingly crucial to making these machine learning systems possible. Layered architectures that embrace mobile, edge and cloud computing complicate where data and code land.

At the confluence of these trends is a need to be able to run machine learning applications in browsers which do not provide direct access to hardware acceleration. There are plans for this to be the case, but until then, Tensorflow.js provides a path forward by leveraging the availability of JavaScript in and out of the browser while getting hardware acceleration where it can. This means the WebGL standard for 3D graphics in the browser and native libraries outside in an environment like Node.js.

This gives us the capacity to move machine learning inference to the edge where many people have powerful GPU capabilities, do not need to push potentially sensitive data to a third party service, and decisions can be localized without the need for the latency of invoking backend services. Beyond these new capabilities, Tensorflow.js represents some very deep thinking about these trends and how to deal with them.

We will cover using pre-trained models that can be fetched remotely as well as the ability to train them directly in the browser. There will be several fun and exciting demonstrations and examples to run and work through.

What you'll learn-and how you can apply it

By the end of this live, hands-on, online course, you’ll understand:

  • What TensorFlow.js is
  • How TensorFlow.js is added to your web application
  • How TensorFlow.js can be used to add machine learning capabilities to your web application
  • How TensorFlow.js takes advantage of Graphical Processing Units (GPUs) directly and indirectly

And you’ll be able to:

  • Build applications using Javascript and the Tensorflow.js library
  • Feed machine learning algorithms with data available to the browser
  • Reuse code in the front and backend

This training course is for you because...

  • You’re a JavaScript developer who is interested in machine learning (ML)
  • You have problems that do not fit solely into conventional backend ML environments
  • You want to gain reuse in ML code across both front and backend environments

Prerequisites

  • Familiarity with Javascript and conventional web development languages, standards and tools.
  • General awareness of machine learning workflows and processes.

Recommended preparation:

  • There is no special setup, but it will require a modern browser that supports WebGL.
  • We will not assume much background in machine learning, but time spent going through existing machine learning courses, such as Getting started with machine learning, will be well spent.

Recommended follow-up:

About your instructor

  • Brian Sletten is a liberal arts-educated software engineer with a focus on forward-leaning technologies. His experience has spanned many industries including retail, banking, online games, defense, finance, hospitality and health care. He has a B.S. in Computer Science from the College of William and Mary and lives in Auburn, CA. He focuses on web architecture, resource-oriented computing, social networking, the Semantic Web, data science, 3D graphics, visualization, scalable systems, security consulting and other technologies of the late 20th and early 21st Centuries.

Schedule

The timeframes are only estimates and may vary according to how the class is progressing

Introduction: Machine Learning Overview (10 minutes)

  • Presentation: Introduction
  • Q&A: We will make sure everyone is comfortable with the high level ideas of machine learning.

Iris Dataset (10 minutes)

  • Presentation: This is the “Hello, World!” of machine learning datasets. We will have a quick introduction to its structure so participants are comfortable with the first few exercises.

Iris: Loading Pre-Existing Models (30 minutes)

  • Presentation
  • Exercise: We will ask the participants to load a pre-existing version of a model trained on the Iris dataset so that it can be used to classify samples locally.
  • Q&A
  • Break: 5 min

Iris: Training in the Browser (35 minutes)

  • Presentation
  • Exercise: We will ask the participants to train a version of the model locally using some default parameters and assess how well it did.
  • Exercise: We will ask the participants to change some of the parameters before retraining the model locally and see how the prediction quality changes.
  • Q&A

Linear Regression (30 minutes)

  • Presentation
  • Exercise: We will ask the participants to train a model on some linear data and make predictions with what is generated.
  • Q&A
  • Break: 5 min

Working with Keras Models (45 minutes)

  • Presentation
  • Exercise: We will ask the participants to load a pre-trained Keras model into the Tensorflow.js library and use it locally.
  • Exercise: We will ask the participants to use the Tensorflow.js Layers API to create Keras-like models locally.
  • Q&A
  • Break: 5 min

Working with TensorFlow Models (45 minutes)

  • Presentation
  • Exercise: We will ask the participants to load and use a pre-trained example model from the Tensorflow.js models repo.
  • Exercise: We will ask the participants to load a pre-trained Tensorflow SavedModel and use it to make predictions locally.
  • Exercise: We will ask the participants to load a pre-trained Tensorflow Hub model and use it to make predictions locally.
  • Q&A: 5 min
  • Break: 5 min

WebRTC + Tensorflow.js (35 minutes)

  • Presentation
  • Exercise: We will ask the participants to use a pre-trained model to interact with their webcams using WebRTC.
  • Q&A