O'Reilly logo
live online training icon Live Online training

Artificial Intelligence for Robotics

Adding decision-making skills to mobile robots

Francis X Govers III

This training in an introduction to advanced AI robotics, and is designed for someone who understands the basics of how to make a robot (like someone who participated in FIRST, or another robot competition), and wants to continue learning about robotics and AI. This is also a good all-around introduction to Artificial Intelligence that is presented in simple, practical terms that pulls out the concepts without a lot of jargon, weird equations, or off the wall examples. While the concepts of AI can be complex, the implementation and usage is not – we are mostly applying very simple rules over and over again.

This is also a good place for you if you are interested in robotics and AI as a career, as we will be using a lot of techniques and principles that professional robotics use for autonomous vehicles, self-driving cars, and unmanned aircraft.

There are (almost) no equations and very little math that you will notice in this course, and we will use a lot of simple illustrations and diagrams. If you wish to take your AI expertise to the next level, this is the one for you!

The course will cover robotics basics (very quickly) to make sure everyone understands motors, controllers, sensors, cameras, speakers, microphones, servo motors and data links. Then we can do some quick robot theory on how to put a really complex robot together using a subsumption architecture and real time control. This gives us a solid foundation to design upon.

At this point we need a quick ROS Tutorial – ROS is the Robotics Operating System, and a great way to put the parts of our robot software together.

Speaking of design, our next section pulls apart robot design and gives an introduction to Systems Engineering, and the art of integration. We will develop use cases, decompose use cases into storyboards, and then write requirements for our software and hardware that gives us a fighting chance to solve problems and anticipate what steps must come later to have a successful robot project.

Now we have a design and a plan. What comes after that?

To have more than just a radio controlled car, we need to give our robot some brains. We need some of that Artificial Intelligence we have all be hearing about, or have even been using in our homes.

We will cover six topics in AI: - Artificial Neural Networks (specifically Convolutional Neural Networks) - Reinforcement Conditioning (Q-learning) - Genetic Algorithms - Path Planning and Navigation - Digital Assistants, Voice commands, Natural Language Processing, and Understanding Intent - Simulation and Artificial Personality

What you'll learn-and how you can apply it

The basic learning objectives can be divided into two parts: Robotics and Artificial Intelligence.

Robotics:

  • What is a robot?
  • Common robot parts.
  • Robot Architectures and Concepts – OODA, Subsumption Architecture
  • Soft Real Time Control System
  • Robotics Operating System (ROS)

Artificial Intelligence:

  • Artificial Neural Network
  • Convolutional Neural Networks
  • Reinforcement Learning
  • Genetic Algorithms
  • Path Planning and Navigation (with and without maps)
  • Digital Assistants, Understanding Intent, Voice Commands

This training course is for you because...

This training is designed for people with an introduction to robots, drones, or unmanned vehicles that want to take their understanding and skills to the next level. You could be a professional engineer or EE already involved in robotics or AI to take advantage of this course. You could be a graduate student wanting to get another view of AI and robotics from a non-academic standpoint.

Prerequisites

You should understand the basics of robotics or have built some sort of Radio Controlled car, plane, or quadcopter, or be familiar with the principles of making a robot, 3D printer, or similar device. It would also help to have a basic understanding of electricity, motors, microcontrollers (Arduino) and Single Board Computers (SBC).

Most of the examples will be using Linux as the operating system – specifically Ubuntu 16. You can use Oracle’s VirtuaBox to run Ubuntu under Windows, which is what the instructor will be doing.

Python – all of the programming examples are done in Python (mostly 2.7 but a couple use Python 3)

Resources:

Govers, Francis X. Artificial Intelligence for Robotics, Packt Publishing 2018.

Materials, downloads, or Supplemental Content needed in advance

We recommend you have Ubuntu Linux loaded on a Virtual Machine (Oracle Virtua Box) or dedicated computer. You may also use a Raspberry PI 3 running either Debian or Ubuntu

Please download and install:

  • Python (should already be installed. I recommend getting the Anaconda distribution at https://www.anaconda.com/download/
  • ROS Kinetic http://wiki.ros.org/kinetic/Installation
  • Mycroft AI digital Voice Assistant https://mycroft.ai/. You will need to create an account
  • Tensorflow https://www.tensorflow.org/
  • Keras Neural Network Frontend https://keras.io/#installation

About your instructor

  • Mr. Govers is an autonomy lead at Bell Helicopter. He is the designer of over 28 unmanned vehicles or robots for land, sea, air and space, including RAMSEE, the autonomous security guard robot. As the deputy Chief Engineer for the US Army’s Future Combat Systems Unmanned Systems, he led the design of seven self-driving cars. He was a participant in the first DARPA Grand Challenge, and was a lead engineer for the International Space Station. He is a US Air Force veteran and a commercial pilot. He has received five outstanding achievement awards from NASA and recognition from Scientific American for “World Changing Ideas”.

Schedule

The timeframes are only estimates and may vary according to how the class is progressing

Day 1

SECTION 1: INTRODUCTION TO AI ROBOTICS (20 mins) - What is a robot? What is Artificial Intelligence? - Subsumption Architecture and OODA – making a foundation - Control Theory, sampling, real time - Soft Real Time Control - Frame Based Scheduling - Questions and Answers Session

LAB 1: BUILD A SOFT REAL TIME, FRAME-BASED CONTROL LOOP (20 mins)

BREAK – 10 MINUTES

SECTION 2: INTRO TO SYSTEMS ENGINEERING FOR ROBOTICS SYSTEMS ENGINEERING (30 mins)

  • Use Cases
  • Developing Storyboards
  • Writing Requirements
  • Extracting Hardware Requirements
  • Testing
  • Questions and Answers Session

LAB 2: DEVELOP USE CASES AND STORYBOARDS FOR A ROBOT TASK (15 mins)

BREAK – 10 MINS

SECTION 3: IMAGE RECOGNITION WITH CONVOLUTIONAL NEURAL NETWORKS (50 mins)

  • Neural Networks – theory and construction
  • Image processing basics – convolution kernels
  • CNN design and theory
  • Training, Learning Rate, Gradient Descent and Back propagation
  • Tensorflow and Keras with Python
  • Preparation for training a Neural Network
  • Testing and Evaluation of Neural Network
  • Common problems – overfitting
  • Deployment on Raspberry Pi 3
  • Questions and Answers Session

BREAK – 10 MINS

LAB 3: BUILD A CONVOLUTIONAL NEURAL NETWORK TO CLASSIFY IMAGES USING KERAS, TENSORFLOW, PYTHON AND A STANDARD DATA SET (60 mins)

Day 2

SECTION 5: BEHAVIOURAL AI WITH REINFORCEMENT LEARNING (50 mins)

  • Machine Learning Concepts
  • Action-State Reinforcement Learning
  • Q-learning applied to behaviors
  • Index States and Actions
  • Improving Performance
  • Questions and Answers Session

BREAK – 10 MINUTES

LAB 5: BUILD REINFORCEMENT LEARNING EXAMPLE LAB (30 mins)

SECTION 6: GENETIC ALGORITHMS (20 mins) - What the heck is a Genetic Algorithm - GA approaches to robotics problems - GA Building Blocks - GA Algorithm walkthrough - Questions and Answer Period

LAB 6: CONSTRUCT GENETIC ALGORITHM WITH TOOLKIT FROM THE BOOK. RUN THROUGH ITERATIONS OF GENERATION SIZE, CROSSOVER, AND MUTATION RATE. (30 mins)

BREAK – 10 MINUTES

SECTION 7: TEACHING YOUR ROBOT TO TALK TO YOU (20 mins)

  • A bit of History: Chatbots, Natural Language Processing Concepts
  • Parts of Speech and Sentence Diagramming
  • The Sticky Problem of Intent
  • Mycroft Open Source Digital Assistant and the DYAI Voice Kit
  • Making a Verbal Skill step by step
  • Questions and Answers

LAB 7: TEACHING YOUR ROBOT TO TELL A JOKE AND TO LISTEN TO A JOKE (30 mins)

BREAK – 10 MINUTES

SECTION 8: ROBOT NAVIGATION (30 mins)

  • What is so hard about indoor navigation?
  • SLAM and Mapmaking
  • Navigation by Landmarks
  • Pathfinding, the A and D Algorithms
  • Floor finding for obstacle avoidance
  • Navigating without a map –learned navigation by Neural Network
  • Questions and Answer Period