Kalman Filtering

Markov Decision Process (MDP)

Simultaneous Localization and Mapping (SLAM)

TensorFlow

Time-of-flight (ToF) camera

Inertial measurement unit (IMU)

State–action–reward–state–action (SARSA)

Asynchronous value iteration

Shared autonomy

Networking

Sensors

Hardware

Camera

Learning to walk

Differential dynamic programming (DDP)

Dynamic programming

Measuring Velocity and Position

Exploration / Exploitation

Linux

Dynamics Modeling

Proportional–integral–derivative (PID) controller

Middleware

Q-learning

Bandits

Quadruped locomotion

Grasping and Manipulation

Interaction-aware learning

Policy Gradient

Reinforcement learning (RL)

Linearly-solvable Markov decision problems

Reward Shaping

Interactive Perception

Partially observable Markov decision process (POMDP)

Learning-based grasping and manipulation

Inverse reinforcement learning

Contractions

Model-based and model-free RL for robot control

Linear–quadratic regulator (LQR)

TD-Gammon

Policy Iteration

Uncertainty

Temporal difference (TD) learning

Imitation learning

Safety

Separation Principle

Reinforcement Learning from Human Feedback (RLHF)

Model Predictive Control

Linear Programming Approach

Neural network

Learning-based perception

Function approximation

CS1951R Introduction to Robotics Brown University Fall 2023 This course offers students the opportunity to build and program an autonomous drone. Focusing primarily on autonomous drones, the course provides a broader insight into modern robotics, encompassing autonomous ground vehicles and robotic arms. Topics include safety, networking, controls, state estimation, and high-level planning. By the end, students can design, build, and operate a robotic drone. The aim of this course is to empower people to build robots. Students will build, program, and fly an autonomous drone. We will cover everything needed to program an autonomous robot, including safety, networking, state estimation, controls, and high-level planning. Although the course focuses on an autonomous drone, we will provide a broad overview of modern robotics, including some topics relating to autonomous ground vehicles and robotic arms. Each student will build and program their own small quadcopter. The course will provide basic components and a small number of replacement parts. After taking this course, students will be able to:

- Explain the space of designs for robotic communications, safety, state estimation, and control.
- Apply that knowledge to construct programs for communications, safety, state estimation, and control.
- Build, program, and operate an autonomous robot drone.
  The class textbook can be found online at [Duckiesky Learning Materials](https://docs.duckietown.org/daffy/doc-sky/out/index.html). Each assignment or project is a link to a specific chapter in this textbook and has corresponding lectures in the edX Edge course.

CS1951R Introduction to Robotics

AA 174B / AA 274B / CS 237B / EE 260B Principles of Robot Autonomy II Stanford University Winter 2023 This course provides a deeper understanding of robot autonomy principles, focusing on learning new skills and physical interaction with the environment and humans. It requires familiarity with programming, ROS, and basic robot autonomy techniques. This course teaches advanced principles for endowing mobile autonomous robots with capabilities to autonomously learn new skills and to physically interact with the environment and with humans. Concepts that will be covered in the course are: Reinforcement Learning (RL) and its relationship to optimal control, contact and dynamics models for prehensile and non-prehensile robot manipulation, as well as imitation learning and human intent inference. Students will learn the theoretical foundations for these concepts. With this course, students will obtain a fundamental understanding of
advanced principles of robot autonomy, including robot learning, physical interaction with
the environment, and interaction with humans. - Familiarity with ROS and basic techniques for robot autonomy (e.g., AA274A or
equivalent).
- Familiarity with programming (e.g., CS 106A or equivalent) and Python.
- College calculus, linear algebra (e.g., CME 100 or equivalent).
- Basic probability and statistics (e.g., CME 106 or equivalent).  There is no required textbook.

AA 174B / AA 274B / CS 237B / EE 260B Principles of Robot Autonomy II

CS 294-40: Learning for robotics and control UC Berkeley Fall 2008 This advanced course focuses on the applications of machine learning in the robotics and control field. It covers a wide range of topics including Markov Decision Processes, control theories, estimation methodologies, and robotics principles. Recommended for graduate students. This is an advanced course in learning for robotics and control. The goal of this course is to help the audience with their research in learning for robotics and control or related topics. A tentative list of topics includes:

- Markov decision processes: value iteration, policy iteration, linear programming, Q learning, TD, value function approximation, inverse reinforcement learning
- Control: linear quadratic regulator, differential dynamic programming, receding horizon / model predictive control
- Estimation: (extended) Kalman filters, particle filters, SLAM
- Robotics: basic principles of various robots, sensors, microcontrollers
- Exploration/Exploitation: bandits, no-regret, e^3  Familiarity with mathematical proofs, machine learning, artificial intelligence, optimization, probability, algorithms, linear algebra; ability to implement algorithmic ideas in code (C/C++ and matlab).

Graduate students only (consent of instructor required for undergraduate students, please talk to me after first lecture and hand me summary of relevant classes/experience so I can decide whether to make an exception). 

CS 294-40: Learning for robotics and control

Robotics is a multidisciplinary field that combines elements of computer science, mechanical and electrical engineering, and more. In the realm of computer science, robotics is often focused on aspects such as robot programming, artificial intelligence (AI), machine learning, and autonomous systems. This can encompass things like perception (how a robot understands its environment), planning (how a robot decides what actions to take), and control (how a robot executes these actions).

Robotics

Computer Science

Robotics

Prerequisites

Common concepts