Robotics Program

Matthew Johnson-Roberson, Director of the Robotics Institute

Cameron Riviere, RI Undergraduate Program Director
Location: NSH 3107

Samantha Bridge, Academic Program Coordinator
Location: NSH 1517
www.ri.cmu.edu/education

Carnegie Mellon University established the first academic department in the United States dedicated to robotics when it created the Robotics Institute in 1979. Since then, it has been a world leader in robotics research and graduate education. Now, starting in the Fall of 2023, it is proud to offer an undergraduate Bachelor of Science in Robotics (BSR).
BSR teaches students how to create, program, and use robotics to solve real world problems. It is an interdisciplinary program taught from the computer science perspective. Students will master core concepts of computer science, mathematical foundations, and algorithmic concepts for robotics. Mechanics and electronics are covered in classes with an emphasis on their use in robotics applications and systems development. Students will acquire a broad knowledge of common robotics components, including software tools, mechanisms, and electronics. Students learn to work effectively in interdisciplinary teams through hands-on projects where they design, prototype, fabricate, and test robotic systems. The curriculum also includes discussions on ethical and societal impacts of robotics, and the development skills to communicate technical ideas to broad audiences.

Students in the B.S. program in Robotics are expected to acquire the following skills upon graduation:

Understand how to use robotics to solve real-world problems, using techniques that include the integration of hardware and software components; designing, modeling, and controlling complex systems; reasoning about sensor data and actuator commands; and using these capabilities for decision making, action selection, and interaction with humans.
Master the core concepts of computer science, with emphasis on programming, computing systems, and algorithm design.
Master the mathematical fundamentals of robotics, including calculus, differential equations, representations of spatial relationships, linear algebra, signal processing, optimization methods, and probability theory.
Master algorithmic concepts for robotics, including sensing and perception algorithms to model and extract information from real-world data; robotic manipulation algorithms to solve multi- axis motion control problems; planning algorithms that find successful paths and trajectories in the presence of obstacles and constraints; and the use of low-level control algorithms to abstract away complex dynamics.
Acquire a broad knowledge of commonly used robotics components and tools, including various sensors, actuators, mechanical components, and software tools such as CAD, ROS, and Matlab; and understand how these components can be integrated into systems to enable effective and safe operation.
Gain hands-on knowledge of prototyping design, fabrication, and testing skills for all aspects of robotics systems, including mechanical, electrical, communication, and computing systems.
Develop the ability to form, manage, and work effectively in interdisciplinary teams using systems thinking, including using block diagrams to decompose complex systems into functional subsystems; working with potential stakeholders to define system requirements; identifying required skillsets to implement subsystems components and creating a suitable team; and assigning and managing task execution and systems integration.
Understand the ethical and societal implications of robotics and artificial intelligence and be able to critically evaluate the impact of these technologies on society.
Develop effective communication and presentation skills, including the ability to communicate complex technical concepts to a broad range of audiences.

B.S. in Robotics Curriculum

Computer Science Core

Computer Science Core (all of the following):		Units
07-128	First Year Immigration Course	3
15-122	Principles of Imperative Computation	12
15-213	Introduction to Computer Systems	12
15-251	Great Ideas in Theoretical Computer Science	12

Robotics core

All of the following:		Units
16-220	Robot Building Practices	12
16-280	General Robotic Systems (if not offered substitute 16-311)	12
16-299	Introduction to Feedback Control Systems	12
16-384	Robot Kinematics and Dynamics	12
16-385	Computer Vision	12
16-450	Robotics Systems Engineering	12
16-474	Robotics Capstone	12
Recommended but not required:
16-170	Concepts of Robotics	5
Course number is being changed to 16-180

ETHICS COURSE

One of the following courses:		Units
16-161	ROB Seminar: Artificial Intelligence and Humanity	12
16-735	Ethics and Robotics	12

mathematics

All of the following Mathematics courses:
15-151	Mathematical Foundations for Computer Science (if not offered, substitute 21-127)	12
16-211	Foundational Mathematics of Robotics	12
21-120	Differential and Integral Calculus	10
21-122	Integration and Approximation	10
21-241	Matrices and Linear Transformations	11
Plus one of the following Probability courses:
15-259	Probability and Computing	12
21-325	Probability	9
36-218	Probability Theory for Computer Scientists	9
36-225	Introduction to Probability Theory	9

Robotics Electives

Three general robotics electives	units
16-3xx and 16-4xx are pre-approved. Elective coursework outside of the Robotics Institute must be approved by the RI Undergraduate Program Director prior to course enrollment.	36

School of Computer Science Electives

One general computer science elective

units

This elective can be from any SCS department; 200-level or above (see exceptions below): Computer Science^15-, Computational Biology^02-, Human Computer Interaction^05-, Machine Learning^10-, Language Technologies^11-, Robotics^16-, and Software Engineering^17-. (NOTE: The following undergraduate courses do NOT count as Computer Science electives: 02-201, 02-223, 02-250, 02-261, 11-423, 15-351, 16-223, 17-200, 17-333, 17-562. Some IDEATE courses and some SCS undergraduate and graduate courses might not be allowed based on course content. Consult with RI Undergraduate Program Director before registration to determine eligibility for this requirement.)

9-12

SCIENCE AND ENGINEERING

All candidates for the bachelor's degree in Robotics must complete a minimum of 36 units offered by the Mellon College of Science and/or the College of Engineering (CIT). These courses offer students an opportunity to explore scientific and engineering domains that can influence their effectiveness as computer scientists upon graduation.

Requirements for this component of the degree are listed under the SCS main page under General Education Requirements.

HUMANITIES AND ARTS

All candidates for the bachelor's degree in Robotics must complete a minimum of 63 units offered by the College of Humanities & Social Sciences and/or the College of Fine Arts. Some courses from the Tepper School of Business also qualify for this requirement. These courses offer students breadth in their education and perspectives and provide students with a better appreciation of social, artistic, cultural, political and economic issues that can influence their effectiveness as computer scientists upon graduation.

Requirements for this component of the degree are listed under the SCS main page under General Education Requirements.

COMPUTING @ CARNEGIE MELLON (1 COURSE)

The following course is required of all students to familiarize them with the campus computing environment:

99-101

Core@CMU

FREE ELECTIVES

A free elective is any Carnegie Mellon course. However, a maximum of nine (9) units of Physical Education and/or Military Science (ROTC) and/or Student-Led (StuCo) courses may be used toward fulfilling graduation requirements.

Double Counting

In general, courses taken in satisfaction of the minor or additional major may also count toward any general education category in the Robotics major (i.e. courses outside of those listed in the Mathematics, Computer Science Core and Robotics Core requirements). Double counting toward Mathematics, Computer Science Core or Robotics Core courses in the Robotics major is strictly limited and depends on the chosen minor (or additional major). In general, students may double count at most five of the above listed requirements toward all other declared additional majors and minors. Additional majors and minors have their own double counting rules as well. Consult with an undergraduate advisor and an advisor from the department of the minor (or additional major) for specific restrictions on double counting.

UNDERGRADUATE RESEARCH THESIS

Robotics majors may use the SCS Honors Research Thesis as part of their degree. The SCS Honors Undergraduate Research Thesis (07-599) typically starts in the fall semester of the senior year, and spans the entire senior year. Students receive a total of 36 units of academic credit for the thesis work, 18 units per semester. Up to 18 units can be counted toward SCS elective requirements (9 per semester for 2 semesters maximum). Students interested in research may also consider using 07-300 Research and Innovation in Computer Science in their junior year since this course will introduce students to various research projects going on in the School of Computer Science that may lead to a senior thesis. This course leads to a subsequent practicum that allows students to complete a small-scale research study or experiment and present a research poster. Students who use the practicum to start their senior thesis can use these units toward the required 36 units.

For more information about the SCS Honors Research Thesis, refer to the SCS Honors Research Thesis section for learning objectives, application requirements and expected outcomes.

Summary of Degree Requirements

Area	Courses	Units
Robotics (core and electives)	11	119
Computer Science (core and SCS elective)	4	46
Mathematics	6	64
Ethics	1	9
Science / Engineering	4	36
Humanities / Arts	7	64
Free Electives	var.	var.
C@CM	1	3
First Year Seminar	1	3
Total Units		360

Sample Course Sequence

The sample given below is for a student who already has credit for introductory programming and one semester of calculus. Students with credit for two semesters of calculus may start with a more advanced math class (e.g. 21-241) in their first year. Students with no credit for introductory programming and/or one semester of calculus will take 15-112 and/or 21-120 in their first semester and shift a few courses to later semesters after consulting with their academic advisor; these students should still be able to complete their degree in four years. It is recommended that students keep their academic load lighter for their Senior Fall semester to account for offsite job interviews or for their Senior Spring semester to account for visits to graduate schools.

First-Year
Fall	Spring
07-128 First Year Immigration Course	16-170 Concepts of Robotics
15-122 Principles of Imperative Computation	15-213 Introduction to Computer Systems
15-151 Mathematical Foundations for Computer Science	21-241 Matrices and Linear Transformations
21-122 Integration and Approximation	Science / Engineering Course
76-101 Interpretation and Argument	Humanities / Arts Elective
99-101 Core@CMU

Second-Year
Fall	Spring
16-220 Intro to Robotic Building Practices	16-299 Introduction to Feedback Control Systems
XX-XXX Probability Course	16-280 General Robotic Systems
16-211 Foundational Mathematics of Robotics	15-251 Great Ideas in Theoretical Computer Science
xx-xxx Humanities / Arts Elective	xx-xxx Humanities / Arts Elective

Third-Year
Fall	Spring
16-384 Robot Kinematics and Dynamics	xx-xxx Robotics Elective
16-385 Computer Vision	xx-xxx Robotics Elective
xx-xxx Humanities / Arts Elective	xx-xxx Humanities / Arts Elective
xx-xxx Humanities / Arts Elective	xx-xxx Humanities / Arts Elective
xx-xxx Free Elective	xx-xxx Free Elective

Fourth-Year
Fall	Spring
16-450 Robotics Systems Engineering	16-474 Robotics Capstone
xx-xxx Ethics Course	xx-xxx SCS Elective
xx-xxx Science / Engineering Course	xx-xxx Science / Engineering Course
xx-xxx Robotics Elective	xx-xxx Free Elective

Minimum number of units required for the degree:360

*The flexibility in the curriculum allows many different schedules, of which the above is only one possibility. Some elective courses are offered only once per year (Fall or Spring). Constrained electives (probability, logic/languages, software systems, artificial intelligence and domains) may be taken in any order and in any semester if prerequisites are met and seats are available. Constrained electives are shown in the specific semesters in the schedule above as an example only. Students should consult with their academic advisor to determine the best elective options depending on course availability, their academic interests and their career goals.

Robotics Program

Robotics Additional Major

Robotics at Carnegie Mellon University focuses on the theme that robotics is both multidisciplinary and interdisciplinary. This means that it draws from many fields, such as mechanical engineering, computer science and electrical engineering, and it also integrates these fields in a novel manner. The foundation of this program lies in motion and control. Upon this base, sensing, cognition, and action are layered. Since robotics involves building artifacts that embody these fundamentals, foci, and systems thinking, there is a "hands-on" course requirement. These foci are brought together by a unique systems perspective special to robotics. Students will complete a capstone course that will tie together previously learned skills and knowledge.

Prerequisites: Successful candidates for the Robotics Additional major will have prerequisite knowledge of C language, basic programming skills, and familiarity with basic algorithms. Students can gain this knowledge by taking 15-122 Principles of Imperative Computation.

Required Courses

There are 10 requirements below in total (one from each category plus two electives):

Overview (one of the following):		Units
16-280	General Robotic Systems	12
16-311	Introduction to Robotics	12

Controls (one of the following)
06-464	Chemical Engineering Process Control	9
16-299	Introduction to Feedback Control Systems	12
18-370	Fundamentals of Control	12
24-451	Feedback Control Systems	12
24-773 - with instructor and Program Director's permission		12

Mechanisms and Manipulation (one of the following options)
16-384	Robot Kinematics and Dynamics	12
OR take both of the following courses:
15-462	Computer Graphics	12
33-141	Physics I for Engineering Students	12

Sensing and Perception (one of the following)
15-387	Computational Perception	9
15-463	Computational Photography	12
16-322	Modern Sensors for Intelligent Systems	12
16-385	Computer Vision	12
16-421	Vision Sensors	12
16-423	Designing Computer Vision Apps	12
85-370	Perception	9
16-722	Sensing and Sensors	12
16-886	Special Topics: Models & Algorithms for Interactive Robotics	12
16-xxx Upper-level RI course with instructor and Program Director’s permission		12

Ethics (one of the following)
16-161	ROB Seminar: Artificial Intelligence and Humanity	12
16-735	Ethics and Robotics	12

Robot Building Practices (one of the following)
16-220	Robot Building Practices	12
16-362	Mobile Robot Algorithms Laboratory	12
16-423	Designing Computer Vision Apps	12
18-349	Introduction to Embedded Systems	12
18-578	Mechatronic Design	12
18-500	ECE Design Experience	12
24-671	Electromechanical Systems Design	12

Systems Engineering
16-450	Robotics Systems Engineering	12
Can be taken concurrently with the Controls requirement.

Capstone Course
16-474	Robotics Capstone	12

Two general robotics electives	Units
16-3xx and 16-4xx are pre-approved. Elective coursework outside of the Robotics Institute must be approved by the RI Undergraduate Program Director prior to course enrollment.	24

Students may count up to 12 units of 16-597 Undergraduate Reading and Research towards the major requirements. A student can also take additional courses from the core; e.g., a student who takes 16-385 as a core can take 16-421 as an elective.

Graduate level Robotics courses may be used to meet elective requirement with permission from the Program Director. Graduate level Mechanical Engineering and Electrical and Computer Engineering courses that are relevant to robotics may be used to meet the elective requirement with permission from the Program Director.

A 3.0 QPA in the Additional Major curriculum is required for graduation. Courses that are taken Pass/Fail or audited cannot be counted for the Additional Major.

Double-Counting Restriction

Students are permitted to double count a maximum of six courses from their Primary Major towards the Additional Major in Robotics. CS Majors are permitted to double count a maximum of five courses from their Primary Major towards the Additional Major in Robotics.

Robotics Minor

http://undergrad.ri.cmu.edu/academics/minor

The Minor in Robotics provides an opportunity for undergraduate students at Carnegie Mellon to learn the principles and practices of robotics through theoretical studies and hands-on experience with robots. The Minor is open to students in any major of any college at Carnegie Mellon. Students initially learn the basics of robotics in an introductory robotics overview course. Additional required courses teach control systems and robotic manipulation. Students also choose from a wide selection of electives in robotics, perception, computer vision, cognition and cognitive science, or computer graphics. Students have a unique opportunity to undertake independent research projects, working under the guidance of Robotics Institute faculty members; this provides an excellent introduction to robotics research for those considering graduate studies.

All Robotics Minors are required to take 16-280 General Robotic Systems (if not offered substitute 16-311). This course is designed to help students understand the big picture of what is going on in robotics through topics such as kinematics, mechanisms, motion planning, sensor based planning, mobile robotics, sensors, and vision. The minor also requires students to take a controls class and a kinematics class. These courses provide students with the necessary intuition and technical background to move on to more advanced robotics courses. In addition to the required courses, students must take 2 electives. The student must have course selection approved by the Director during the application submission process.

A 2.5 QPA in the Minor curriculum is required for graduation. Courses that are taken Pass/Fail or audited cannot be counted for the Minor.

Admission

Admission to the Undergraduate Minor in Robotics is limited to current Carnegie Mellon students. Students interested in signing up for the minor should fill out the application form available on the program website.

Prerequisite

Successful candidates for the Robotics Minor will have prerequisite knowledge of C language, basic programming skills, and familiarity with basic algorithms. Students can gain this knowledge by taking 15-122 Principles of Imperative Computation.

Required Courses

Overview (one of the following):		Units
16-280	General Robotic Systems	12
16-311	Introduction to Robotics	12

Controls (choose one of the following):
06-464	Chemical Engineering Process Control	9
16-299	Introduction to Feedback Control Systems (Computer Science)	12
18-370	Fundamentals of Control	12
24-451	Feedback Control Systems	12
24-773	Multivariable Linear Control	12
16-xxx	Upper-level RI course with instructor and Program Director's permission	12

Mechanisms and Manipulation (choose one of the following options):
16-384	Robot Kinematics and Dynamics	12
OR take both of the following courses:
15-462	Computer Graphics	12
33-141	Physics I for Engineering Students	12

Robot Building Practices (choose one of the following)
16-220	Robot Building Practices	12
16-362	Mobile Robot Algorithms Laboratory	12
16-423	Designing Computer Vision Apps	12
18-349	Introduction to Embedded Systems	12
18-500	ECE Design Experience	12
18-578	Mechatronic Design	12
24-671	Electromechanical Systems Design	12

Electives

Two general robotics electives	Units
16-3xx and 16-4xx are pre-approved. Elective coursework outside of the Robotics Institute must be approved by the RI Undergraduate Program Director prior to course enrollment.	24

Graduate level Robotics courses may be used to meet the elective requirement with permission from the Program Director. Graduate level Mechanical Engineering and Electrical and Computer Engineering courses that are relevant to robotics may be used to meet the elective requirement with permission from the Program Director.

Students may count up to 12 units of 16-597 Undergraduate Reading and Research towards the minor requirements.

Double-Counting Restriction

Courses being used to satisfy the requirements for the Robotics Minor may not be counted towards another minor. Students are permitted to double count a maximum of two courses from their Major (excluding General Education requirements) towards the Minor in Robotics. Free electives are not subject to the double counting policy.

Robotics Courses

About Course Numbers:

Each Carnegie Mellon course number begins with a two-digit prefix that designates the department offering the course (i.e., 76-xxx courses are offered by the Department of English). Although each department maintains its own course numbering practices, typically, the first digit after the prefix indicates the class level: xx-1xx courses are freshmen-level, xx-2xx courses are sophomore level, etc. Depending on the department, xx-6xx courses may be either undergraduate senior-level or graduate-level, and xx-7xx courses and higher are graduate-level. Consult the Schedule of Classes each semester for course offerings and for any necessary pre-requisites or co-requisites.

16-161 ROB Seminar: Artificial Intelligence and Humanity: Fall and Spring: 12 units
In 1965 British mathematician I.J. Good wrote, An ultraintelligent machine could design even better machines; there would then unquestionably be an intelligence explosion, and the intelligence of man would be left far behind. As we enter an age where companies like Uber are testing driverless cars in Pittsburgh and innovative interfaces like IBMs Watson can play jeopardy and learn techniques for medical diagnoses, how are we to negotiate an intelligence explosion that for many individuals might threaten the very notions of what it means to be human? The future of human-to-machine relationships will likely define our historical epoch and yet, many young technologists and humanists underestimate the downstream impact of technological innovations on human society. Presently, we have little choice but to attend to this rapidly anxiety-ridden question. This seminar will attend to the challenge of present existential questions on what it means to be human (read not machine) in the context of a rapidly advancing technological age. We will consider human narratives throughout history that exam how governments and individual citizens defined humanity in the context of slavery and colonialism as a framework for exploring and projecting what it means to be human in the age of rapidly advancing intelligent machines. We will trace the technological advancements of the recent five decades and identify historical precedents and speculative narratives that help us to consider issues like labor, economic disparity, negotiations of power, human dignity and ethical responsibility within the context of human relations with advancing technological tools that are now coined, artificial intelligence.

16-170 Concepts of Robotics: Spring: 5 units
The course will introduce students to the main foundational concepts and techniques used in robotics including perception, cognition, and action. Concepts will be grounded in a range of real-world robotic systems to highlight the use of common robotics components such as sensor selection, sensor processing and fusion, path planning algorithms, mechanism design, reasoning about interactions with the environment, and systems integration. Applications of robotics will be discussed along with methods for mapping application requirements to design choices for robotic systems. Students will also be introduced to ethical issues surrounding robotics, including considerations around potential future of uses of robotics technologies. The course will contain programming and written assignments designed to give students a feel for the practical aspects of robot sensing, planning, and actuation.
Prerequisite: 15-112 Min. grade C

16-211 Foundational Mathematics of Robotics: Fall: 12 units
This course will cover core mathematics concepts used in many advanced robotics courses at the RI. Perhaps unlike prior courses in math, the focus of this class will be to ground concepts in robotics algorithms or applications. For example: How to move and manipulate objects in 3D space (coordinate transforms, rotations). How to move an articulated robots end-effector in Cartesian space (Jacobians, gradient optimization). How to have a robot learn to recognize a vision input (neural networks, back propagation). How to plan navigate a robot optimally (dynamic programming, A* Search).
Prerequisites: 21-122 and 21-241

16-220 Robot Building Practices: Fall: 12 units
This course is designed to provide students with a comprehensive set of mechanical and electronics skills required for designing, prototyping, building, and troubleshooting robotic systems. Students will learn about basic robotic components and how to obtain, build, or fix them to create functional robotic systems. The course will cover mechanical skills specific to robotics, including sketching, 3D CAD modeling, 3D printing, laser cutting and other machine shop tools. Students will also learn the fundamentals of circuit design, breadboarding, and PCB layout using CAD tools, as well as how to use measurement equipment and soldering techniques. They will gain hands-on experience with motor controllers and microcontrollers, essential components for controlling robots. The class project will give students the opportunity to learn how to design and implement power transmission systems and prototype mechanical components required in building a functional robotic module and later a full robotic system. The course will include robot-specific topics such as kinematics, robot actuators, sensors, and perception algorithms. Upon completion of this course, students will have a solid foundation in electronics and mechanical prototyping for robotics and be able to create innovative robots for a variety of applications.
Prerequisite: 15-122 Min. grade C

16-223 IDeATe Portal: Creative Kinetic Systems: Fall: 10 units
The art and science of machines which evoke human delight through physical movement is founded on a balance of form and computation. This introductory physical computing course addresses the practical design and fabrication of robots, interactive gadgets, and kinetic sculptures. The emphasis is on creating experiences for human audiences through the physical behavior of devices which embody computation with mechanism, sensing, and actuation. Specific topics include basic electronics, elementary mechanical design, embedded programming, and parametric CAD. A key objective is gaining an intuitive understanding of how information and energy move between the physical, electronic, and computational domains to create a compelling behavior. The final projects are tested in the field on children and adults. This interdisciplinary course is an IDeATe Portal Course open to students from all colleges. For students choosing to follow an IDeATe program it is an entry into either Physical Computing or Intelligent Environments. The structure of the class revolves around collaborative exercises and projects which introduce core physical computing and system engineering techniques in a human-centric context. Students apply system and design thinking across multiple domains, work together to make and test several devices, and participate in wide-ranging critique which considers both technical and artistic success.

Course Website: https://courses.ideate.cmu.edu/16-223

16-224 IDeATe: Re-Crafting Computational Thinking with Soft Technologies: Spring: 12 units
This course focuses on teaching introductory concepts of Robotics, Mechatronics, and Computer Science using an arts-based approach. During the course, students will build their own weaving robot, program it, and learn how weaving art is connected to computer programming and matrix mathematics. Students will also learn the history of weaving, how to design beautiful patterns, and how to extract the features of those patterns into mathematical equations and computer programs.

16-235 Fantastic Robots and How to Fold Them: Spring: 9 units
This course will focus on the basics of robotics through a hands-on approach. Students will build their own robots by designing a mechanical structure and embedding actuators, sensors, and controllers. They will then use these robots to solve a simple maze with obstacles. The course content will be delivered through lectures, workshops, and a course-long team project. In classical robotics, we explore the three main behaviors of robots through the work frame of "sense-plan-act". Robots are more than just these behaviors, and students will learn about how to make the physical embodiments of robots through an overview of design and manufacturing techniques for robot mechanisms. Students will be able to make their own mechanisms, improve the system through hardware or software, and learn how to analyze the kinematics and dynamics of these mechanisms to understand and control the motion.
Prerequisites: 15-110 Min. grade C or 15-112 Min. grade C or 15-104 Min. grade C

16-264 Humanoids: Spring: 12 units
This course surveys perception, cognition, and movement in humans, humanoid robots, and humanoid graphical characters. Application areas include more human-like robots, video game characters, and interactive movie characters.

Course Website: http://www.cs.cmu.edu/~cga/humanoids-ugrad/

16-299 Introduction to Feedback Control Systems: Spring: 12 units
This course is designed as a first course in feedback control systems for computer science majors. Course topics include classical linear control theory (differential equations, Laplace transforms, feedback control), linear state-space methods (controllability/observability, pole placement, LQR), nonlinear systems theory, and an introduction to control using computer learning techniques. Priorities will be given to computer science majors with a robotics major or minor.
Prerequisites: 21-122 and 15-122
Course Website: http://www.cs.cmu.edu/~cga/controls-intro/

16-311 Introduction to Robotics: Spring: 12 units
This course presents an overview of robotics in practice and research with topics including vision, machine learning, motion planning, mobile mechanisms, kinematics, inverse kinematics, and sensors. In course projects, students construct LEGO robots which are driven by a microcontroller, with each project reinforcing the basic principles developed in lectures. Students usually work in teams of three: an electrical engineer, a mechanical engineer, and a computer scientist. Groups are typically self-formed except for the first lab. This course will also expose students to some of the contemporary happenings in robotics, including current robotics research, applications, robot contests and robots in the news. Students registering for this course must register for both Mon/Wed mornings and Tuesday afternoon sections.
Prerequisites: 18-202 Min. grade C or 21-240 Min. grade C or 21-241 Min. grade C or 24-311 Min. grade C or 21-260 Min. grade C

Course Website: http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/16311/www/current/

16-322 Modern Sensors for Intelligent Systems: Spring: 12 units
The class aims at introducing sensing technologies for robots and other intelligent systems. The course will cover the physical principles of traditional sensors, sensor calibration and evaluation, signal processing algorithms for different sensors, and examples of sensor applications for robots or other intelligent systems. On the sensing system design part, the course will cover the common sensor fusion design and algorithms, and provide examples of sensing systems for different robots or intelligent systems, such as wearable sensors, self-driving cars, autonomous vehicles, assistant robots, and field robots in extreme conditions. The class will contain lectures, two lab sessions, and a course project.

16-350 Planning Techniques for Robotics: Spring: 12 units
Planning is one of the core components that enable robots to be autonomous. Robot planning is responsible for deciding in real-time what should the robot do next, how to do it, where should the robot move next and how to move there. This class does an in-depth study of popular planning techniques in robotics and examines their use in ground and aerial robots, humanoids, mobile manipulation platforms and multi-robot systems. The students learn the theory of these methods and also implement them in a series of programming-based projects. To take the class students should have taken an Intro to Robotics class and have a good knowledge of programming and data structures.

Course Website: http://www.cs.cmu.edu/~maxim/classes/robotplanning/

16-362 Mobile Robot Algorithms Laboratory: Fall: 12 units
This course is an introduction to the theory and algorithms of multirotor vehicle autonomy. Students will work individually to develop a multirotor simulator in Python and C++, add sensors, plan, and perform exploration. Lectures will cover topics to advance the capabilities of the simulator. Homeworks will be designed to increase the autonomy capabilities of the multirotor vehicle. The class will culminate in an individual project that pushes the autonomy capabilities developed throughout the course and may cover multi-robot aerial autonomy, dynamic environment modeling, or advanced planning and control. In order to succeed in the course, students must have a 2nd year science/engineering level background in mathematics (matrices, vectors, coordinate systems) and have already mastered at least one object-oriented programming language like C++ or Python. When the course is over, students will have written a single software system that has been incrementally extended in functionality and regularly debugged throughout the semester.

16-371 Personalized Responsive Environments: Spring: 9 units
[IDeATe collaborative course]. Environmental factors have a significant impact on mood and productivity. Creating responsive environments necessitates the design of surroundings that are able to metamorphose in order to optimize user strengths and available resources and evolve in stride with user needs. This course will investigate the development of spaces that adapt to user preferences, moods, and task specific demands. Both the design and engineering of such personalized environments will be explored. Central course concepts will include, understanding the user, integrating various modalities (e.g., light, heat, sound) to support the changing needs of task and user, and the creation of adaptive environments that learn user preferences over time. Please note that there may be usage/materials fees associated with this course.
Prerequisites: 60-223 Min. grade C or 62-150 Min. grade C or 18-090 Min. grade C or 15-104 Min. grade C

16-374 IDeATe: Art of Robotic Special Effects: Spring: 12 units
Inspired by the early "trick" films of George Melies, this project-oriented course brings together robotics and film production technique to infuse cinema with the wonder of live magic. Students will learn the basics of film production using animatronics, camera motion control, and compositing. The projects apply these techniques to create innovative physical effects for short films, all the way from concept to post-production. The course emphasizes real-time practical effects to explore the immediacy and interactivity of improvisation and rehearsal. The robotics topics include animatronic rapid prototyping and programming human-robot collaborative performance. The course includes a brief overview of the history of special effects and robotics to set the work in context.

Course Website: https://courses.ideate.cmu.edu/16-374

16-375 IDeATe: Robotics for Creative Practice: Fall: 10 units
Robots come in all shapes and sizes: it is the integration of software and hardware that can make any machine surprisingly animate. This project-oriented course brings art and engineering together to build performance systems using embodied behavior as a creative medium. Students learn skills for designing, constructing and programming automated systems for storytelling and human interaction, then explore the results through exhibition and performance. Technical topics include closed-loop motion control, expressive physical and computational behavior, machine choreography, and performance conceptualization. Discussion topics include both contemporary kinetic sculpture and robotics research. This interdisciplinary course is part of IDeATe Physical Computing but is open to any student.
Prerequisites: 15-110 or 15-104 or 15-112 or 99-361 or 60-212 or 60-210

Course Website: https://courses.ideate.cmu.edu/16-375

16-376 IDeATe: Kinetic Fabrics: Spring: 10 units
Kinetic Fabrics brings together the fields of robotics and textiles to explore their unified creative and expressive potential. It is a wide-open frontier for kinetic art, wearable art, and architectural installation. In this course students will build a variety of performative systems combining fabrics and robotic technologies. Students will apply modular actuation and sensing to textile artworks, using software designed to facilitate fluid explorations, rapid iterations, and playful experimentation. Students will learn basic textile skills, such as hand and machine sewing, as well as gain facility and familiarity with the characteristics of multiple type of fabrics. Historical precedents as well as contemporary examples of works will support students creative growth and knowledge of the field. Students' course work will include short-term and long-term projects, sampling and prototyping, critique, and documentation. Additionally, students will organize an end-of-semester event where they will perform a developed kinetic fabric work for a public audience.

Course Website: https://courses.ideate.cmu.edu/16-376

16-384 Robot Kinematics and Dynamics: Fall: 12 units
Foundations and principles of robotic kinematics. Topics include transformations, forward kinematics, inverse kinematics, differential kinematics (Jacobians), manipulability, and basic equations of motion. Course also include programming on robot arms.
Prerequisites: 15-122 Min. grade C or 16-311 or 18-202 or 21-241 or 24-311

16-385 Computer Vision: Fall and Spring: 12 units
This course provides a comprehensive introduction to computer vision. Major topics include image processing, detection and recognition, geometry-based and physics-based vision, sensing and perception, and video analysis. Students will learn basic concepts of computer vision as well as hands on experience to solve real-life vision problems. This course is for undergraduate students only.
Prerequisites: (18-202 Min. grade C and 15-122 Min. grade C) or (21-259 Min. grade C and 15-122 Min. grade C and 21-241 Min. grade C) or (21-241 Min. grade C and 21-256 Min. grade C and 15-122 Min. grade C) or (24-282 Min. grade C and 15-122 Min. grade C and 21-241 Min. grade C) or (21-254 Min. grade C and 15-122 Min. grade C and 21-241 Min. grade C)

Course Website: http://www.cs.cmu.edu/~16385/

16-397 Art, Conflict and Technology: Spring: 12 units
This course considers the period of violence in Northern Ireland from 1968 to 1998 known as The Troubles, and recent issues pertaining to sovereignty and borders caused by Brexit, Britain's proposed exit from the European Union, as a point of comparison between societies rife with strife, division and predilections to violence. We investigate the ways in which visual art to literature to theatrical performance explores and interrogates societal conflict and emergence from conflict, and how evolving technological systems influence political power dynamics and modes of artistic practice. We will use the legacy of societal conflict in Ireland and Northern Ireland to compare concepts and physical manifestations of borders, barriers and bridges in the region and in global contexts. We will examine fluctuating development of democratic processes in Ireland and Northern Ireland, individual and group public performance, and the influence of technologically crude and highly sophisticated tools on communities emerging from strife. We will use our analytical lens to focus on figurative and literal borders, barriers and bridges to explore work produced in Belfast, Derry and Dublin, alongside circumstances and artistic practice in present-day Pittsburgh, Cuidad Juarez, Jerusalem and Soweto. On a visit to Ireland and Northern Ireland over spring break, students will meet with artists, writers, legislators, community organizers, academics and ex-combatants, to learn about their past experience and current motivations. Students will analyze artistic practice, peacekeeping initiatives and performance of identity in relation to the historical framework from which it emerges in Ireland and Northern Ireland. We will use this foundation as a point of comparison to practices throughout the world. Students will process their experience and developing analytical skills by documenting their responses through original creative work.

16-421 Vision Sensors: Spring: 12 units
This course covers the fundamentals of vision cameras and other sensors - how they function, how they are built, and how to use them effectively. The course presents a journey through the fascinating five hundered year history of "camera-making" from the early 1500's "camera obscura" through the advent of film and lenses, to today's mirror-based and solid state devices (CCD, CMOS). The course includes a significant hands-on component where students learn how to use the sensors and understand, model and deal with the uncertainty (noise) in their measurements. While the first half of the course deals with conventional "single viewpoint" or "perspective" cameras, the second half of the course covers much more recent "multi-viewpoint" or "multi-perspective" cameras that includes a host of lenses and mirrors.
Prerequisites: 21-111 and 21-241
Course Website: http://www.cs.cmu.edu/~ILIM/courses/vision-sensors/

16-423 Designing Computer Vision Apps: Fall: 12 units
Computer vision is a discipline that attempts to extract information from images and videos. Nearly every smart device on the planet has a camera, and people are increasingly interested in how to develop apps that use computer vision to perform an ever expanding list of things including: 3D mapping, photo/image search, people/object tracking, augmented reality etc. This course is intended for students who are not familiar with computer vision, but want to come up to speed rapidly with the latest in environments, software tools and best practices for developing computer vision apps. No prior knowledge of computer vision or machine learning is required although a strong programming background is a must (at a minimum good knowledge of C/C++). Topics will include using conventional computer vision software tools (OpenCV, MATLAB toolboxes, VLFeat, CAFFE), and development on iOS devices using mobile vision libraries such as GPUImage and fast math libraries like Armadillo and Eigen. For consistency, all app development will be in iOS and it is expected that all students participating in the class have access to an Intel-based MAC running OS X Mavericks or later. Although the coursework will be focussed on a single operating system, the knowledge gained from this class is intended to generalize to other mobile platforms such as Android etc.
Prerequisites: (21-240 and 15-213) or (21-241 and 15-213) or (18-202 and 18-213)

Course Website: http://16423.courses.cs.cmu.edu

16-425 Medical Image Analysis: Spring: 12 units
Students will gain theoretical and practical skills in 2D, 3D, and 4D biomedical image analysis, including skills relevant to general image analysis. The fundamentals of computational medical image analysis will be explored, leading to current research in applying geometry and statistics to segmentation, registration, visualization, and image understanding. Additional and related covered topics include de-noising/restoration, morphology, level sets, and shape/feature analysis. Students will develop practical experience through projects using the latest version of the National Library of Medicine Insight Toolkit ( ITK ) and SimpleITK, a popular open-source software library developed by a consortium of institutions including Carnegie Mellon University and the University of Pittsburgh. In addition to image analysis, the course will include interaction with radiologists and pathologist(s). *** Lectures are at CMU and students will visit clinicians at UPMC. Some or all of the class lectures may also be videoed for public distribution, but students may request to be excluded from distributed video. 16-725 is a graduate class, and 16-425 is a cross-listed undergraduate section. 16-425 is new this year, and has substantially reduced requirements for the final project and for the larger homework assignments, nor does it require shadowing the clinicians. Prerequisites: Knowledge of vector calculus, basic probability, and either C++ or python, including basic command-line familiarity and how to pass arguments to your own command-line programs. Extensive expertise with C++ and templates is not necessary, but some students may find it helpful.

Course Website: http://www.cs.cmu.edu/~galeotti/methods_course/

16-441 Advanced CP/SIS: Urban Intervention: Fall and Spring: 12 units
This course introduces students to theories, practices, and communities for critical investigation of urban spaces and play within them. The course unfolds along two parallel trajectories: research (literature review, lectures, readings, demonstrations) and design (three iterated individualized projects and a fourth larger scale final project). The first half of the course will introduce students to a wide range of theories and techniques within urban intervention that draw from fluxus, the situationist international, activism and hacktivism, as well as public policy, philosophy, psychology and economics. Students will study theoretical and practical frameworks for artistic intervention into public urban spaces, while concurrently researching actual sites and communities within Pittsburgh for experimentation. Students are required to conceptualized projects on larger (urban) scales, and find ways to implement their projects safely and legally by pursuing the necessary administrative, social, technical, financial steps required to create meaningful interventions in public spaces. This class will specifically explore three media for urban intervention: Sound Outdoor video projection Robotics, Autonomy and Mobility in the way of remote control vehicles (e.g. cars, quad-copters, etc.). For each theme, students are required to produce one project that is iterated twice or more. The undergraduate (60441) and graduate (60741) sections of the course meet concurrently and follow the same syllabus and assignments. In addition to the coursework documented in the syllabus, Graduate level students are expected to write a research paper suitable for submission to a notable relevant academic conference. This process includes a rough draft, revisions and a completed and formatted paper ready for submission

16-450 Robotics Systems Engineering: Fall: 12 units
Systems engineering examines methods of specifying, designing, analyzing and testing complex systems. In this course, principles and processes of systems engineering are introduced and applied to the development of robotic devices. The focus is on robotic system engineered to perform complex behavior. Such systems embed computing elements, integrate sensors and actuators, operate in a reliable and robust fashion, and demand rigorous engineering from conception through production. The course is organized as a progression through the systems engineering process of conceptualization, specification, design, and prototyping with consideration of verification and validation. Students completing this course will engineer a robotic system through its compete design and initial prototype. The project concept and teams can continue into the Spring-semester (16-474 Robotics Capstone) for system refinement, testing and demonstration.
Prerequisites: 16-311 Min. grade B and (16-299 Min. grade B or 18-370 Min. grade B or 24-451 Min. grade B)

16-455 IDeATe: Human-Machine Virtuosity: Spring: 12 units
[IDeATe course] Human dexterous skill embodies a wealth of physical understanding which complements computer-based design and machine fabrication. This project-oriented course explores the duality between hand and machine through the practical development of innovative design and fabrication systems. These systems fluidly combine the expressivity and intuition of physical tools with the scalability and precision of the digital realm. Students will develop novel hybrid design and production workflows combining analog and digital processes to support the design and fabrication of their chosen projects. Specific skills covered include 3D modeling (CAD), 3D scanning, algorithmic geometric modeling, digital and robotic fabrication (additive and subtractive manufacturing), motion capture and computer based sensing, and human-robot interaction design. Areas of interest include architecture, art, and product design.

Course Website: https://courses.ideate.cmu.edu/16-455

16-456 Reality Computing Studio: Fall: 12 units
[IDeATe collaborative course] Reality computing encompasses a constellation of technologies focused around capturing reality (laser scanning, photogrammetry), working with spatial data (CAD, physical modeling, simulation), and using data to interact with and influence the physical world (augmented / virtual reality, projector systems, 3d printing, robotics). Taught in collaboration with the school of architecture, this studio asks students to apply these technologies to real world problems such as residential design, sustainability, and infrastructure monitoring.

Course Website: http://ideate.cmu.edu/about-ideate/departments/robotics-institute/reality-computing/

16-457 Reality Computing II: Spring: 12 units
[IDeATe collaborative course] Reality computing encompasses a constellation of technologies focused around capturing reality (laser scanning, photogrammetry), working with spatial data (CAD, physical modeling, simulation), and using data to interact with and influence the physical world (augmented / virtual reality, projector systems, 3d printing, robotics). This iteration of the reality computing course will focus on "design realization": the translation from digital design to fully realized tangible artifact. Collaborating with the UDBS design studio, and within the context of a full-scale residential prototype, students will investigate how reality computing technologies can be used to accelerate and advance the process of design realization by using reality computing to understand existing homes, map design data into the real world, and highlight conflicts between design and reality. Topics of special focus within the course are residential design (John Folan) and augmented reality and robotics (Pyry Matikainen).

Course Website: http://ideate.cmu.edu/about-ideate/departments/robotics-institute/reality-computing/

16-461 Experimental Capture: Fall: 9 units
Performance capture is used in applications as varied as special effects in movies, animation, sports training, physical rehabilitation, and human-robot/human-computer interaction. This course will survey state-of-the-art techniques and emerging ideas, in the industry and in academia, to capture, model, and render human performances. The course will be a mix between lectures and discussion of recent progress in human motion capture and analysis. The course evaluation will be project-based, in which students will capture their own body and face motion, and build projects around the data they collect individually and as a group. We will cover: 1. Capture Techniques: We will describe and use various systems including motion capture, video-based capture, depth sensors, scanners, and eye-gaze trackers; 2. Modeling and Representation: We will cover classic and contemporary representations of face and body pose and motion, including statistical and physics-based techniques; 3. Rendering Applications: As new rendering paradigms emerge, new applications continue to develop. We will study recent progress in animation, synthesis, classification, and rehabilitation on new forms of displays. Please note that there may be usage/materials fees associated with this course.
Prerequisites: 60-422 or 15-365

16-465 Game Engine Programming: Spring: 10 units
This course is designed to help students understand, modify, and develop game engines. Game engines consist of reusable runtime and asset pipeline code. They provide game-relevant abstractions of low-level system services and libraries, making it easier to write bug-free games that work across multiple platforms. Game engines also handle artistic content, providing or integrating with authoring tools to ease the process of creating high-fidelity games. In this course, we will discuss the problems game engines attempt to solve, examine how current state-of-the-art engines address these problems, and create our own engines based on what we learn. We will cover both the content authoring and runtime aspects of engines. Coursework will consist of frequent, tightly-scoped programming and system design assignments; expeditions through game engine source code; and two group projects and #8212; one in an engine created from scratch, and one that requires modification of an existing engine. Prerequisites: Students will be expected to be fluent in at least one programming language. We will be working with C++, Javascript, and a smattering of Python. We will be using git for version control and code sharing. The assignments in the course will be designed to be completed on an OSX or Linux workstation (e.g. the IDeATe "virtual cluster"). Working with Windows will be possible, but might require extra effort. We will be building a 3D model pipeline around Blender, but no prior knowledge of the tool will be assumed.
Prerequisites: 62-150 Min. grade C or 15-213 Min. grade C or 15-104 Min. grade C or 15-112 Min. grade C

16-467 Introduction to Human Robot Interaction: Spring: 12 units
The field of human-robot interaction (HRI) is fast becoming a significant area of research in robotics. The basic objective is to create natural and effective interactions between people and robots. HRI is highly interdisciplinary, bringing together methodologies and techniques from robotics, artificial intelligence, human-computer interaction, psychology, education, and other fields. This course is primarily lecture-based, with in-class participatory mini-projects, homework assignments, a group term project that will enable students to put theory to practice, and a final. The topics covered will include technologies that enable human-robot interactions, the psychology of interaction between people and robots, how to design and conduct HRI studies, and real-world applications such as assistive robots. This course has no prerequisites, but some basic familiarity with robots is recommended (programming knowledge is not necessary, but is useful for the term project).

Course Website: http://harp.ri.cmu.edu/courses

16-469 Innovation and Shared Prosperity: Community-engagement for change: Fall: 12 units
How might we, as a community of learners, utilize our collective talents for innovation and shift our society towards greater justice? In this course we will cover the historical and social context of university-community engagement, discuss best practices in engagement efforts, and operationalize emergent strategy alongside design justice principles. Learnings from this course will foster the growth of lifelong dispositions and habits that can empower learners to chart a course for their personal careers that are consonant with community empowerment and societal equity. This class is for individuals interested in pursuing a career at the intersection of technology and societal equity or for individuals who are interested in issues of justice and equity more broadly. Learning methods for this course will include readings, reflections, and in-class discussions. Students in this class will be asked to draw on their own experiences and to explore case studies. We also anticipate that students will directly engage with local community as facilitated through existing connections with the Center for Shared Prosperity.

16-474 Robotics Capstone: Spring: 12 units
In this course students refine the design of, build, integrate, test, and demonstrate the robot they designed in the prerequisite Systems Engineering course (16-450). The students are expected to continue to apply the process and methods of systems engineering to track requirements, evaluate alternatives, refine the cyberphysical architectures, plan and devise tests, verify the design, and validate system performance. The course consists of lectures, class meetings, reviews, and a final demonstration. Lectures cover special topics in project management. During class meetings the students and instructor review progress on the project and discuss technical and project-execution challenges. There are three major reviews, approximately at the end of each of the first three months of the semester. For each review, students give a presentation and submit an updated version of the System Design and Development Document. The course culminates in a System Performance Validation Demonstration at the end of the semester. Students also hold a special demonstration of their robotic system for the broader Robotics community.
Prerequisite: 16-450 Min. grade C

16-480 IDeATe: Creative Soft Robotics: Spring: 10 units
This experimental course offers unique topics situated at the intersection of robotics research and the arts, with a specific research focus that varies each semester. In this course, students survey the state of an emerging research area, then design and fabricate experimental systems and artworks on the theme. Students are guided through literature search and technical paper analysis to identify opportunities and techniques. The textual study spans contemporary robotics and arts literature. The project component is research-focused and explores novel techniques in design, fabrication, programming, and control. The project sequence culminates in the collaborative design of expressive robotic systems which match technical innovation with a human need or artistic expression. The initial iteration of the course focuses on soft robotics, an emerging discipline centered on devices constructed from compliant materials that incorporate sensing and actuation. The literature survey spans soft robotics and kinetic sculpture. The projects center on fabricating forms that incorporate actuators and sensors using silicone rubber cast into 3D-printed and laser-cut molds. This course is offered by IDeATe and this iteration will satisfy minor requirements for IDeATe Soft Technologies or IDeATe Physical Computing.

Course Website: https://courses.ideate.cmu.edu/16-480

16-595 Undergraduate Independent Study: All Semesters
For students to pursue an independent study with a Robotics Institute faculty member.

16-597 Undergraduate Reading and Research: All Semesters
Undergraduate Reading and Research enables students to gain academic credits for conducing independent studies in robotics. Students must work with a robotics faculty advisor to devise a specific objective, activities (such as reading, evaluating, designing, coding, building, or testing robotic systems) and metrics for evaluation of their performance by their advisor.

16-621 MSCV Project I: Spring: 12 units
The MSCV capstone project course is designed to give project teams additional feedback on their capstone project from peers and faculty. Every week, capstone teams will present their project PPFs (Past-Present-Future) reports. For the presenting teams, the capstone course will help develop presentation and communication skills. For the students participating as peer-reviewers, it will help develop critical thinking and the ability to give constructive advice.

Course Website: https://piazza.com/cmu/spring2019/16621

16-622 MSCV Capstone: Fall: 12 units
The MSCV capstone project course is designed to give project teams additional feedback on their capstone project from peers and faculty. Every week, capstone teams will present their project PPFs (Past-Present-Future) reports. For the presenting teams, the capstone course will help develop presentation and communication skills. For the students participating as peer-reviewers, it will help develop critical thinking and the ability to give constructive advice.

16-623 Advanced Computer Vision Apps.: Fall: 12 units
Computer vision is a discipline that attempts to extract information from images and videos. Nearly every smart device on the planet has a camera, and people are increasingly interested in how to develop apps that use computer vision to perform an ever expanding list of things including: 3D mapping, photo/image search, people/object tracking, augmented reality etc. This course is intended for graduate students who are familiar with computer vision, and are keen to learn more about the applying state of the art vision methods on smart devices and embedded systems. A strong programming background is a must (at a minimum good knowledge of C/C++), topics will include using conventional computer vision software tools (OpenCV, MATLAB toolboxes, VLFeat, CAFFE, Torch 7), and development on iOS devices using mobile vision libraries such as GPUImage, Metal and fast math libraries like Armadillo and Eigen. For consistency, all app development will be in iOS and it is expected that all students participating in the class have access to an Intel-based MAC running OS X Mavericks or later. Although the coursework will be focused on a single operating system, the knowledge gained from this class will easily generalize to other mobile platforms such as Android etc.
Prerequisites: 16-385 or 16-720
Course Website: http://16623.courses.cs.cmu.edu

16-627 MSCV Seminar: Fall
(Only open to MSCV students.) MSCV students will be required to participate in this one-semester seminar course which will prepare them for the MSCV project starting in the Spring semester. The first part of this course will cover talks by computer vision and related faculty about the ongoing research, development programs related to Computer Vision at CMU. The second part of this course will include student/faculty tutorial on topics such as OpenCV, Dataset Creation, Mechanical Turk etc. The goal of this series is to get students acquainted with practical knowledge for a successful project. In the last month of the course, each lecture will cover upto four possible MSCV projects pitched by faculty or industrial sponsors. At the end of the course students will turn in their choices, and a faculty committee will assign them the final projects.

16-633 Special Topic: Robot Cognition and Learning: Spring: 12 units
This is open to both Grad and Undergrad students. This project class focuses on developing cognitive and learning systems for robots. Students will become familiar with and use state of the art software tools to build prototype systems, as well as how to evaluate these systems. The course project will involve implementing a cognitive/learning system on a real robot. For undergraduates, this course is an elective for the Robotics Major.

Course Website: https://www.cs.cmu.edu/~cga/cog/

16-639 Special Topic: Scalable Robotic Systems: Infrastructure Development/Deployment: Spring: 12 units
As robots continue to permeate various sectors such as healthcare, manufacturing, and autonomous transportation, the ability to build and scale robust robotic systems has become increasingly crucial. This course aims to bridge the gap between single-robot deployments and large-scale robotic systems, providing students with a comprehensive understanding of the processes and tools necessary to scale their robotic projects.

16-663 F1Tenth Autonomous Racing: Spring: 12 units
This hands-on, lab-centered course is for senior undergraduates and graduate students interested in the fields of artificial perception, motion planning, control theory, and applied machine learning. It is also for studentsinterested in the burgeoning field of autonomous driving. This course introduces the students to the hardware, software and algorithms involved in building and racing an autonomous race car. Every week, students take two lectures and complete an extensive hands-on lab. By Week 6, the students will have built, programmed and driven a 1/10th scale autonomous race car. By Week 10, the students will have learned fundamental principles in perception, planning and control and will race using map-based approaches. In the last 6 weeks, they develop and implement advanced racing strategies, computer vision and machine learning algorithms that will give their team the edge in the race that concludes the course.

16-664 Self-Driving Cars: Perception & Control: Fall: 12 units
This course will teach the theoretical underpinnings of self-driving car algorithms and the practical application of the material in hands-on labs. Topics will include deep learning, computer vision, sensor fusion, localization, trajectory optimization, obstacle avoidance, and vehicle dynamics.

16-665 Robot Mobility on Air, Land, & Sea: Fall: 12 units
Required core course for MRSD first-year students. Many robots are designed to move through their environments. Three prevalent environments on earth are land, air, and water. This course will explore the modeling, control, and navigation of ground-based (wheeled and legged), air-based (rotorcraft such as quadcopters), and water-based robots.

16-667 Autonomous Air Vehicle Design and Development: All Semesters: 12 units
OPEN TO GRADUATE AND UNDERGRADUATE STUDENTS; Autonomous Air Vehicles are finding new applications in Civil Air Transportation and Emergency Response scenarios. They carry passengers and valuable supplies and must be certified to operate in both urban and rural areas, close to people, buildings, highways, mountains, and dense forest canopies. This presents significant challenges to perception, control systems, and navigation through austere environments. The design limits and flight operations of the aircraft must be understood to be certified by the FAA. In Autonomous Air Vehicle Development, students will design, develop, and test prototype autonomous aircraft for specific missions defined by the new HeroX GoAero Challenge. This multidisciplinary course will go from concept to test and challenge students to apply sound theoretic approaches to a practical design. Students will learn how to design and build resilient autonomous air vehicle systems and the challenges of real-world design, operations, certification, and testing.

Course Website: https://www.herox.com/goaero

16-675 Manufacturing Futures: Spring: 12 units
The course will introduce an array of technologies that will contribute to the future of making things and will be organized into 4 logical modules that will culminate in a team-based design project. Module 1 (Manufacturing Visions and Design Methodolgy): David Bourne. Module 2 (Manufacturing Processes and Process Tradeoffs): Brandon Bodily. Module 3 (Electronic Manufacturing): Rahul Panat. Module 4 (Workforce Development) : David Bourne.

16-682 Robotic Systems Development Project Course II: Fall: 15 units
Required core course for MRSD second-year students. This course is the second semester in a two-semester sequence intended to enable student teams to design and implement robot systems from the requirements development phase through implementation, verification, and demonstration of a working prototype. Teams of 4-5 students continue work on a project provided by industrial and academic partners, refine design requirements, refine or create new subsystems, and integrate and demonstrate the full system.

16-714 Advanced Control for Robotics: Fall: 12 units
This course will discuss advanced control algorithms that can make robots behave more intelligently. This course is directed to studentsprimarily graduate although talented undergraduates are welcome as wellinterested in advanced control.
Prerequisite: 16-711 Min. grade C

16-715 Advanced Robot Dynamics and Simulation: Fall: 12 units
This course explores the fundamental mathematics behind modeling the physics of robots, as well as state-of-the-art algorithms for robot simulation. We will review classical topics like Lagrangian mechanics and Hamilton's Principle of Least Action, as well as modern computational methods like discrete mechanics and fast linear-time algorithms for dynamics simulation. A particular focus of the course will be rigorous treatments of 3D rotations and non-smooth contact interactions (impacts and friction) that are so prevalent in robotics applications. We will use numerous case studies to explore these topics, including quadrotors, fixed-wing aircraft, wheeled vehicles, quadrupeds, humanoids, and manipulators. Homework assignments will focus on practical implementation of algorithms and a course project will encourage students to apply simulation methods to their own research.

16-720 Computer Vision: Fall and Spring: 12 units
Section A is a required core course for MRSD first-year students, and Section B is a required core course for MSCV students. This course introduces the fundamental techniques used in computer vision, that is, the analysis of patterns in visual images to reconstruct and understand the objects and scenes that generated them. Topics covered include image formation and representation, camera geometry, and calibration, computational imaging, multi-view geometry, stereo, 3D reconstruction from images, motion analysis, physics-based vision, image segmentation and object recognition. The material is based on graduate-level texts augmented with research papers, as appropriate. Evaluation is based on homeworks and a final project. The homeworks involve considerable Python programming exercises. Texts recommended but not required: Title: "Computer Vision Algorithms and Applications" Author: Richard Szeliski Series: Texts in Computer Science Publisher: Springer ISBN: 978-1-84882-934-3 Title: "Computer Vision: A Modern Approach" Authors: David Forsyth and Jean Ponce Publisher: Prentice Hall ISBN: 0-13-085198-1

Course Website: http://www.andrew.cmu.edu/course/16-720/

16-725 (Bio)Medical Image Analysis: Spring: 12 units
Students will gain theoretical and practical skills in 2D, 3D, and 4D biomedical image analysis, including skills relevant to general image analysis. The fundamentals of computational medical image analysis will be explored, leading to current research in applying geometry and statistics to segmentation, registration, visualization, and image understanding. Additional and related covered topics include de-noising/restoration, morphology, level sets, and shape/feature analysis. Students will develop practical experience through projects using the latest version of the National Library of Medicine Insight Toolkit ( ITK ) and SimpleITK, a popular open-source software library developed by a consortium of institutions including Carnegie Mellon University and the University of Pittsburgh. In addition to image analysis, the course will include interaction with radiologists and pathologist(s). *** Lectures are at CMU and students will visit clinicians at UPMC. Some or all of the class lectures may also be videoed for public distribution, but students may request to be excluded from distributed video. 16-725 is a graduate class, and 16-425 is a cross-listed undergraduate section. 16-425 is new this year, and has substantially reduced requirements for the final project and for the larger homework assignments, nor does it require shadowing the clinicians. Prerequisites: Knowledge of vector calculus, basic probability, and either C++ or python, including basic command-line familiarity and how to pass arguments to your own command-line programs. Extensive expertise with C++ and templates is not necessary, but some students may find it helpful.

Course Website: http://www.cs.cmu.edu/~galeotti/methods_course/

16-726 Learning-based Image Synthesis: Spring: 12 units
This course introduces machine learning methods for image and video synthesis. The objectives of synthesis research vary from modeling statistical distributions of visual data, through realistic picture-perfect recreations of the world in graphics, and all the way to providing interactive tools for artistic expression. Key machine learning algorithms will be presented, ranging from classical learning methods (e.g., nearest neighbor, PCA) to deep learning models (e.g., ConvNets, NeRF, deep generative models, including GANs, VAEs, autoregressive models, and diffusion models). Finally, we will discuss image and video forensics methods for detecting synthetic content. In this class, students will learn to build practical applications and create new visual effects using their own photos and videos.

16-730 Robotics Business: Spring: 12 units
This course introduces and develops business concepts that will be useful to new and existing companies, while focusing on robotic technology exemplars. The concepts begin with how to identify a new idea to for a business that can be effectively started. Initial ideas often start as a grandiose plan to change the world and these plans are legitimately the fuel that drive new businesses forward. However, when a company starts (e.g., builds a prototype or writes a first line of code), what is the least product a company can produce that customers still want and need? This kernel and #8212; extracted from the "big plan" and #8212; is a Minimal Viable Product (MVP). Once an MVP business kernel is formulated, we will learn and study how to understand customer needs, how to market a new idea and how raise and manage money for a new business entity. These steps abridge information that can be found in an MBA curriculum, but engineers and scientists focused on the technical side will need this information to participate in the process of building companies. In parallel, we will investigate the marketplace through the stock market. The stock market is a powerful window into the world of business. In other words, when a new business is built it has to live inside the competitive environment of every other business. To understand this eco-system, we will follow several companies in-situ as they go through their own ups-and-downs within the business world. The course is project based. Each student will either build their own business concept, or they will build an improvement plan that would be targeted to improve an existing business. Professor Bourne is a founding member of the Robotics Institute(1979) and has taught business concepts within the Tepper Business School and the Robotics Institute since 1988. In addition, he is the President of his own company Design One Software.

16-735 Ethics and Robotics: Intermittent: 12 units
This course contextualizes robotics, AI, and machine learning within cultural conversation, ethics, and power relationships in society. It will draw upon "AI and Humanity" as well as numerous other texts, including Mindless by Simon Head, Drone Theory by Gr and #233;goire Chamayou, and news articles. The course will culminate in team-based design and futuring project addressing the ways in which robotic technologies will influence society and values in the near future. Our target audience is students who will participate in computer science and robotics research and can use this course to inform future research and career decisions.

Course Website: https://vdean.github.io/16-735-ethics-robotics.html

16-737 Special Topic: Research to Startup: creating a startup from robotics research: Intermittent: 12 units
(This course is offered only to Ph.D. students in SCS, or with instructor permission.) This course is for Ph.D. students interested in exploring turning their research into a startup. Advances in AI and robotics have opened exciting opportunities for robotics-based startups. But with that comes challenges. In this class, students will form small teams to take an idea based in part on their research and work through the early steps of converting it into a company. This will require taking a dispassionate view of your research as a product or service and assessing its market and value. Each team will work through customer discovery, vetting ideas, creating and communicating a vision and strategy, fundraising, and building a product. We will have guest lectures and discussions where we will learn from the experiences of people who have created startups. We will learn about CMU resources to help with IP and technology transfer and discuss open-source strategies. We will discuss leveraging Ph.D. students' honed research skills to be successful company founders and #8212;e.g., related work investigations, presentations, and time management. We will emphasize the difference between research and commercial software and provide tools, technologies, and methodologies for a software development lifecycle. We cannot cover everything you need to know in one semester, but we will expose you to the essential aspects necessary to get started.

16-740 AI for Manipulation: Spring: 12 units
Manipulation is the process of changing the state of objects through direct physical interactions. To perform manipulation tasks in unstructured environments, autonomous robots will need to learn about the objects in their surroundings as well as the skills required to manipulate and change the state of these objects. In this course, we explore the use of machine learning and data-driven algorithms for robot manipulation. The course introduces students to the wide variety of challenges posed by manipulation tasks, and how these challenges can be formulated as learning problems. Students are taught how these problems can be solved using machine learning techniques. The types of machine learning methods covered in this course include supervised, unsupervised, active, and reinforcement learning methods. The course includes both lectures and guided paper discussions.

16-741 Mechanics of Manipulation: Fall: 12 units
Mechanics of Manipulation is a graduate level course that dives into the fundamentals of robotic manipulation. Through this course you will learn the kinematics, statics, and dynamics of robotic manipulators as they interact with the world to accomplish tasks. You will gain experience with the intelligent use of kinematic constraint, gravity, and frictional forces. Additional topics include rigid body mechanics, automatic planning based on mechanics, deformable manipulation, and simulation of dynamic manipulation. Applications of robotic manipulation are drawn from physical human-robot interaction, manufacturing, and other domains.

Course Website: http://www.cs.cmu.edu/afs/cs/academic/class/16741-s07/www/index.html

16-742 Geometry of Locomotion: Fall: 12 units
This course introduces geometric methods for the analysis of locomoting systems. Focusing on the kinematics of locomoting systems, the course covers topics from differential geometry, geometric mechanics, and motion planning . Specific topics include configuration spaces, manifolds, groups, Lie groups, representations of velocity, holonomic and nonholonomic constraints, constraint curvature, response to cyclic inputs and distance metrics. The primary goal of this class is to develop an intuitive understanding of these concepts and how they are used in locomoting systems, rather than working through a set of formal proofs and derivations. We do, however, incorporate enough mathematical formalism for this class to serve as a starting point for further investigation into this topic area. We also call upon biological data, when available, and relate to the mathematical formalisms in the class.

16-745 Optimal Control and Reinforcement Learning: Spring: 12 units
This is a course about how to make robots move through and interact with their environment with speed, efficiency, and robustness. We will survey a broad range of topics from nonlinear dynamics, linear systems theory, classical optimal control, numerical optimization, state estimation, system identification, and reinforcement learning. The goal is to provide students with hands-on experience applying each of these ideas to a variety of robotic systems so that they can use them in their own research.

Course Website: http://www.cs.cmu.edu/~cga/dynopt/

16-748 Underactuated Robots: Fall: 12 units
People and animals move through and interact with the world in a fundamentally dynamic way. In the vast majority of cases the same cannot be said for robots. In fact, many conventional approaches to motion planning and robot control attempt to explicitly cancel out the dynamics associated with different tasks. This class will consider underactuated robots, systems that do not have full control over their state and therefore cannot be planned for or controlled via conventional methods. Our goal will be to make novel locomoting robots act more "naturally." This class will highlight the relationship between conventional ideas from deterministic motion planning and control design (e.g., dynamic programming and linear-quadratic regulators) and their contemporary counterparts, many of which help form the analytical basis for the probabilisitic reasoning that underlies contemporary AI systems (e.g., POMDPs). Note that this course is inspired by and, for the most part, will follow the format of "Underactuated Robotics: Learning, Planning, and Control for Efficient and Agile Machines" created by Prof. Russ Tedrake at MIT. We will take several tangents, but the course materials provided by Prof. Tedrake through MIT Open Courseware are an incredible resource for this course (and really just in general).

16-761 Mobile Robots: Spring: 12 units
The course is targeted to graduate level students. The lectures will develop the fundamentals for enabling autonomy of multi rotor aerial vehicles. Students will individually complete assignments related to autonomous quadrotor flight, including motion planning, control, dynamics, state estimation, and perception. The class will culminate in a final project in which students may work together in groups or individually to enhance the autonomy capabilities developed through the assignments.

Course Website: https://mr-cmu.github.io

16-762 Mobile Manipulation: Spring: 12 units
In this project-based course, you'll learn about mobile manipulation through hands-on experience working with real mobile manipulators. You'll gain experience with teleoperation, autonomy, perception, navigation, manipulation, and human-robot interaction, all within the context of mobile manipulators. You'll also learn about robot design, collaborative research, and applications for mobile manipulators.

16-765 Robotics & AI for Agriculture: Spring: 12 units
Robotics and artificial intelligence technologies have the potential to increase the efficiency, long-term sustainability, and profitability of agricultural production methods. This class will introduce common aspects of agricultural systems, the AI/Robotics tools that are being used to address them, and key research challenges looking forward. Technical topics include IoT sensor networks, in-field computer vision, 3D crop mapping and modeling, mobile robot navigation, and robotic manipulation of plants. Course sessions will be split evenly between lectures by the instructor and student-led discussion of relevant papers from the contemporary research literature.

16-778 Mechatronic Design: Spring: 12 units
Mechatronics is the synergistic integration of mechanism, electronics, and computer control to achieve a functional system. This course is a semester-long multidisciplinary capstone hardware project design experience in which small (typically four-person) teams of electrical and computer engineering, mechanical engineering and robotics students deliver an end-of-course demonstration of a final integrated system capable of performing a mechatronic task. Throughout the semester, the students design, configure, implement, test and evaluate in the laboratory devices and subsystems culminating in the final integrated mechatronic system. Lectures will complement the laboratory experience with comparative surveys, operational principles, and integrated design issues associated with the spectrum of mechanism, microcontroller, electronic, sensor, and control components.

Course Website: http://www.ece.cmu.edu/courses/items/18578.html

16-782 Planning and Decision-making in Robotics: Fall: 12 units
Planning and Decision-making are critical components of autonomy in robotic systems. These components are responsible for making decisions that range from path planning and motion planning to coverage and task planning to taking actions that help robots understand the world around them better. This course studies underlying algorithmic techniques used for planning and decision-making in robotics and examines case studies in ground and aerial robots, humanoids, mobile manipulation platforms and multi-robot systems. The students will learn the algorithms and implement them in a series of programming-based projects.

16-785 Integrated Intelligence in Robotics: Vision Language Planning: Intermittent: 12 units
This is a project-oriented course that covers interdisciplinary topics on cognitive intelligence in robotic systems. Cognitive abilities constitute high-level, humanlike intelligence that exhibits reasoning or problem-solving skills. Such abilities as semantic perception, use of language, and task planning can be built on top of low-level robot autonomy. The topics covered generally bridge across multiple technical areas, for example, vision-language intersection and language-action/plan grounding. The project theme in Spring 2023 is "movie making" that presents various robotics and machine learning challenges ranging from content generation such as scenario generation or scene/video synthesis/editing to robotics automation such as autonomous camera control or autonomous stop-motion control. This course is composed of 50% lectures and 50% seminar classes. The course objectives will also put a special emphasis on learning research skills, e.g., problem formulation, literature review, ideation, evaluation planning, results analysis, and hypothesis verification. The course is discussion intensive, and thus attendance is required.

Course Website: http://www.cs.cmu.edu/~jeanoh/16-785/

16-791 Applied Data Science: Spring: 12 units
This course explores the rapidly developing field of data science in the context of its pragmatic applications. Applied Data Science strives to achieve three main goals. The first is to optimize the efficacy of decision making by human managers. The second is to maximize the utilization of available data, so that no important clue is ever missed. The third is to improve understanding of data and the underlying processes that produce it. This course aims at building skills required to systematically achieve those goals in practice. The students will gain and solidify awareness of the most prevalent contemporary methods of Data Science, and develop intuition needed for assessing practical utility of the studied topics in application scenarios. They will be able to learn how to formulate analytic tasks in support of project objectives, how to define successful analytic projects, and how to evaluate utility of existing and potential applications of the discussed technologies in practice.

16-792 Applied Machine Learning: Intermittent
This course explores the rapidly developing field of machine learning in the context of its pragmatic applications. The domain of Applied Machine Learning strives to achieve three main goals. The first is to build effective models to optimize the efficacy of decision-making. The second is to maximize the utilization of available data so that no important clue is ever missed. The third is to gain or improve an understanding of data and the underlying processes that produce it. Students are required to register for 9 units to receive credit for lectures but may also register for 12 units which will include 3 units of capstone project.

16-820 Advanced Computer Vision: Fall: 12 units
16-820 is a required core course for MSCV students and is intended to move at a slightly faster pace compared to 16-720. This course introduces the fundamental techniques used in computer vision, that is, the analysis of patterns in visual images to reconstruct and understand the objects and scenes that generated them. Topics covered include camera geometry and calibration, multi-view stereo, 3D reconstruction, image detection, segmentation, and tracking, and physics-based vision. The homeworks involve considerable Python programming exercises.

16-823 Physics-based Methods in Vision (Appearance Modeling): Intermittent: 12 units
Everyday, we observe an extraordinary array of light and color phenomena around us, ranging from the dazzling effects of the atmosphere, the complex appearances of surfaces and materials, and underwater scenarios. For a long time, artists, scientists, and photographers have been fascinated by these effects, and have focused their attention on capturing and understanding these phenomena. In this course, we take a computational approach to modeling and analyzing these phenomena, which we collectively call "visual appearance". The first half of the course focuses on the physical fundamentals of visual appearance, while the second half of the course focuses on algorithms and applications in a variety of fields such as computer vision, graphics and remote sensing and technologies such as underwater and aerial imaging.
Prerequisites: 16-385 or 16-720 or 15-462 or 16-820
Course Website: http://www.cs.cmu.edu/afs/cs/academic/class/16823-f06/

16-824 Visual Learning and Recognition: Spring: 12 units
This graduate-level computer vision course focuses on representation and reasoning for large amounts of data (images, videos, associated tags, text, GPS locations, etc.) toward understanding the visual world surrounding us. We will be reading an eclectic mix of classic and recent papers on topics including Theories of Perception, Mid-level Vision (Grouping, Segmentation, Poses), Object and Scene Recognition, 3D Scene Understanding, Action Recognition, Multimodal Perception, Language and Vision Models, Deep Generative Models, Efficient Neural Networks, and more. We will cover a wide range of supervised, semi-supervised, self-supervised, and unsupervised approaches for each topic above.
Prerequisites: 15-781 Min. grade B or 16-720 Min. grade B or 16-722 Min. grade B or 10-701 Min. grade B or 16-385 Min. grade B

Course Website: https://visual-learning.cs.cmu.edu/

16-825 Learning for 3D Vision: Spring: 12 units
Any autonomous agent we develop must perceive and act in a 3D world. The ability to infer, model, and utilize 3D representations is therefore of central importance in AI, with applications ranging from robotic manipulation and self-driving to virtual reality and image manipulation. While 3D understanding has been a longstanding goal in computer vision, it has witnessed several impressive advances due to the rapid recent progress in (deep) learning techniques e.g. differentiable rendering, single-view 3D prediction. The goal of this course is to explore this confluence of 3D Vision and Learning-based Methods.

16-831 Introduction to Robot Learning: Fall and Spring: 12 units
Robots need to make sequential decisions to operate in the world and generalize to diverse environments. How can they learn to do so? This is what we call the "robot learning" problem and it spans topics in machine learning, visual learning and reinforcement learning. In this course, we will learn the fundamentals of topics in machine/deep/visual/reinforcement-learning and how such approaches are applied to robot decision making. We will study fundamentals of: 1) machine (deep) learning with emphasis on approaches relevant for cognition, 2) reinforcement learning: model-based, model-free, on-policy (policy gradients), off-policy (q-learning), etc.; 2) imitation learning: behavior cloning, dagger, inverse RL and offline RL.; 3) visual learning geared towards cognition and decision making including topics like generative models and their use for robotics, learning from human videos, passsive internet videos, language models; and 4) leveraging simulations, building differentiable simulations and how to transfer policies from simulation to the real world; 5) we will also briefly touch topics in neuroscience and psychology that provide cognitive motivations for several techniques in decision making. Throughout the course, we will look at many examples of how such methods can be applied to real robotics tasks as well as broader applications of decision making beyond robotics (such as online dialogue agents etc.). The course will provide an overview of relevant topics and open questions in the area. There will be a strong emphasis on bridging the gap between many different fields of AI. The goal is for students to get both the high-level understanding of important problems and possible solutions, as well as low level understanding of technical solutions. We hope that this course will inspire you to approach problems in congnition and embodied learning from different perspectives in your research. (As of 3/21/2023)

Course Website: https://docs.google.com/document/d/1Lx2IkUMvtETH52ZMl7eySZX3yLWQnED746ET9g_tte0/edit?usp=sharing

16-833 Robot Localization and Mapping: Spring: 12 units
Robot localization and mapping are fundamental capabilities for mobile robots operating in the real world. Even more challenging than these individual problems is their combination: simultaneous localization and mapping (SLAM). Robust and scalable solutions are needed that can handle the uncertainty inherent in sensor measurements, while providing localization and map estimates in real-time. We will explore suitable efficient probabilistic inference algorithms at the intersection of linear algebra and probabilistic graphical models. We will also explore state-of-the-art systems.

Course Website: http://frc.ri.cmu.edu/~kaess/teaching/16833/Spring2018

16-845 Insects and Robots: Fall: 12 units
This course will cover all facets of modeling, design, fabrication, and analysis of robots operating on the insect scale, with a microrobotics perspective. Insects can perform different tasks, such as manipulation or locomotion, with their small scale bodies varying from 200m to 16cm length. Similarly, we can define a micro-robotic system as an autonomous or semi-autonomous device with features on the micron scale or that make use of micron-scale physics for mobility or manipulation of objects. Due to their small size scales, microrobots will encounter difficulties unlike their macro-scale counterparts, in terms of fabrication and autonomy. In this project-based course, our aim will be on learning the physics of scaling, fabrication paradigms, actuation and sensing strategies, with numerous case studies, and to build an insect-inspired robotic system. We will also discuss multiple applications such as surgical robotics, mobile microrobots, multi-agent systems, and micro/nano manipulation.

16-848 Hands: Design and Control for Dexterous Manipulation: Spring: 12 units
Research related to hands has increased dramatically over the past decade. Robot hand innovation may be at an all time high, with new materials and manufacturing techniques promoting an explosion of ideas. Hands have become a priority in virtual reality and telepresence. Even the study of how people use their hands is seeing the growth of new ideas and themes. With all of this attention on hands, are we close to a breakthrough in dexterity, or are we still missing some things needed for truly dexterous manipulation? In this course, we will survey robotic hands and learn about the human hand with the goal of pushing the frontiers on hand design and control for dexterous manipulation. We will consider the necessary kinematics and dynamics for dexterity, what sensors are required to carry out dexterous interactions, the importance of reflexes and compliance, the role of machine learning in grasping and manipulation, and the challenge of uncertainty. We will explore state of the art manufacturing and design techniques, including innovations in soft robotics and embedded sensing. We will examine the human hand: its structure, sensing capabilities, human grasp choice and control strategies for inspiration and benchmarking. Students will be asked to present one or two research papers, participate in discussions and short research or design exercises, and carry out a final project.

Course Website: http://graphics.cs.cmu.edu/nsp/course/16899-s18/

16-855 Special Topics: Tactile Sensing and Haptics: Spring: 12 units
Touch is an important perception modality for both humans and robots. This course aims at providing an overview of the touch perception system for both robots and humans, and provide students with some hands-on experience with the popular touch sensors and devices. On the side of robot sensing, the course will cover the topics on the working principles and designs of robot touch sensors, signal processing algorithms for tactile sensing, and the application of tactile sensing in different robotic tasks; on the side of haptics, the course will introduce the neurological and cognitive study in human haptic system, and the designs and applications of haptic devices that provide a human-machine interface. The human-machine interface is a core part of Virtual Reality (VR) and teleoperation of robots when touch is involved. The course includes lectures, research paper presentation and discussion, and course projects with tactile sensors or haptic devices.

16-873 Spacecraft Design-Build-Fly Laboratory: Fall and Spring: 12 units
Spacecraft design is a truly interdisciplinary subject that draws from every branch of engineering. This course integrates broad skillsets from mechanical engineering, electrical and computer engineering, computer science, and robotics toward the goal of designing, building, testing, and flying a small spacecraft over the course of two semesters. Students will engage directly in all aspects of the spacecraft mission lifecycle from initial requirements definition through mission operations. YES, WE ARE REALLY GOING TO LAUNCH A SATELLITE INTO SPACE AS PART OF THIS COURSE. Students will work in subsystem teams, each focusing on some aspect of the spacecraft, but will be exposed to many different disciplines and challenges. Practical, hands-on, engineering skills will be emphasized, along with building and testing physical hardware and flight software.

16-874 Spacecraft Design-Build-Fly Laboratory 2: Spring: 12 units
(ENROLLMENT IS BY INSTRUCTOR APPROVAL ONLY) This course is a continuation of 16-/18-873, and together these two courses make a sequence culminating in the launch of the satellite designed and built over two consecutive semesters. Spacecraft design is a truly interdisciplinary subject that draws from every branch of engineering. This course integrates broad skillsets from mechanical engineering, electrical and computer engineering, computer science, and robotics toward the goal of designing, building, testing, and flying a small spacecraft over the course of two semesters. In this, the second semester of the two-semester sequence, students will work in subsystem teams to fabricate spacecraft components and finally integrate them into a complete spacecraft by the end of the semester. YES, WE ARE REALLY GOING TO LAUNCH A SATELLITE INTO SPACE AS PART OF THIS COURSE. Practical, hands-on, engineering skills will be emphasized, along with building and testing physical hardware and flight software.

16-878 Advanced Mechatronic Design: Fall: 12 units
This course is designed for students who have a background in mechatronics by having taken a mechatronics design course or through practice. The course will be a combination of laboratories and lectures and will culminate in a class project. The topics covered will be microcontroller hardware subsystems: timer systems, PWM, interrupts; analog circuits, operational amplifiers, comparators, signal conditioning, interfacing to sensors, actuator characteristics and interfacing; C language features for embedded software, register level programming, hardware abstraction layers, event driven programming, state machines, state charts.

16-879 Medical Robotics: Fall: 12 units
This course presents an overview of medical robotics intended for graduate students and advanced undergraduates. Topics include robot kinematics, registration, navigation, tracking, treatment planning, and technical and medical aspects of specific applications. The course will include guest lectures from robotics researchers and surgeons, as well as observation of surgical cases. The course is open to non-majors who have the requisite background.

16-880 Special Topics: Engineering Haptic Interfaces: Spring: 12 units
This course focuses on addressing challenges in the field of haptics from an engineer's perspective. We will begin by studying human haptic perception and an introduction into psychophysics. We will then study the design and control of haptic systems which provide touch feedback to a user. The class format will include lectures, discussion, paper presentations, laboratories and assignments using hardware that will be shipped to the students, and a class project. This class is designed to be a graduate/advanced undergraduate course and requires a background in dynamic systems, mechatronics, and basic programming. Mechanical prototyping, robotics, and feedback control knowledge are useful skills for this class but are not required.

16-881 Seminar Deep Reinforcement Learning for Robotics: Spring: 12 units
Deep RL has a lot of promise to teach robots how to choose actions to optimize sequential decision-making problems, but how can we make deep RL work in the real world? This is a seminar course in which we read papers related to deep learning for robotics and analyze the tradeoffs between different approaches. We will read mostly state-of-the-art papers that were very recently published (e.g. recent CoRL, RSS), but we will also look at some older papers that use different approaches. The goals of the course are to 1) understand what is needed to make deep learning work for robotics 2) analyze the tradeoffs between different approaches. Each class, 2 papers will be presented. These papers will both achieve a similar robotics task but will use different learning-based approaches. The class will discuss these papers and try to understand the strengths and limitations of the approach described in each paper. The list of papers that we will be discussing this year is still to be determined; please see the website for the list of papers that we have used in past semesters: https://sites.google.com/view/16-881-cmu/paper-lists?authuser=0 The seminar is a great followup course to 16-831, 16-884, 10-403, or 10-703.

Course Website: https://sites.google.com/view/16-881-cmu/home?authuser=0

16-882 Systems Engineering and Applied Robotics: Spring: 12 units
This course is intended for graduate students of all disciplines who are interested in learning about Systems Engineering and its application in the development of interdisciplinary technical systems. The first part of the course introduces students to the models, methods, and techniques of Systems Engineering. The second part of the course is a study on the adaptation of Systems Engineering in the development of novel robotic systems in applied fields. Each student in the class will perform a semester long study on a special topic in Systems Engineering and a critical evaluation of the process in the development of innovative robotics for applications in space, mining, agriculture, mining, and others.

16-883 Special Topics: Provably Safe Robotics: Spring: 12 units
Safe autonomy has become increasingly critical in many application domains. It is important to ensure not only the safety of the ego robot, but also the safety of other agents (humans or robots) that directly interact with the autonomy. For example, robots should be safe to human workers in human-robot collaborative assembly; autonomous vehicles should be safe to other road participants. For complex autonomous systems with many degrees of freedom, safe operation depends on the correct functioning of all system components, i.e., accurate perception, optimal decision making, and safe control. This course deals with both the design and the verification of safe robotic systems. From the design perspective, we will talk about how to assure safety through planning, prediction, learning, and control. From the verification perspective, we will talk about verification of deep neural networks, safety or reachability analysis for closed loop systems, and analysis of multi-agent systems.

Course Website: http://www.cs.cmu.edu/~cliu6/provably-safe-robotics.html

16-884 Deep Learning for Robotics: Fall: 12 units
The goal of this course is to study relevant topics towards building intelligent robots that can learn to act and perceive in the real world. The course material should be a self-contained collection of key topics from the intersection of four research areas geared towards this common goal: a) Robot Learning and amp; Deep RL; (b) Computer Vision; (c) Control; (d) Psychology and amp; Neuroscience. This course is geared mainly towards learning and brainstorming. There will be two classes every week. In this first class, instructor will present an in-depth overview of a topic, and then in the second class, students will present instructor-assigned papers related to that topic. There will be no homeworks and just a course project. In the first quarter, we will cover state-of-the-art topics in robot learning (deep RL, inverse RL, etc.) and control (optimal control, dynamic movement primitives, etc.) by studying classical and recent papers in the area. In the second quarter, we will study the role of perception in control and vice-versa to build methods that can learn from high-dimensional raw sensory input. In the third quarter, we will discuss the state of the current understanding of how the brain integrates action and perception. We will also discuss relevant papers from ontogeny (child development literature in Psychology) and phylogeny (evolutionary development literature in Biology) of biological animals that have inspired ideas in learning and robotics. Finally, in the fourth quarter, we will bring these ideas together to brainstorm potential high-level directions that could guide the development of intelligent robots.

16-885 Special Topics: Tactile Sensing and Haptics: Fall: 12 units
Touch is an important perception modality for both humans and robots. This course aims at providing an overview of the touch perception system for both robots and humans. On the side of robot sensing, the course will cover the designs of robot touch sensors, signal processing algorithms for tactile sensing, and the application of tactile sensing in different robotic tasks; on the side of haptics, the course will introduce the neurological and cognitive study in the human haptic system, and the designs and applications of haptic devices that provide a human-machine interface. The course incorporates lectures, research paper presentations, and discussion. The combination of different modules aims to present both the basics and state-of-art research directions in the field.

16-886 Special Topics: Models & Algorithms for Interactive Robotics: Spring: 12 units
Robot interaction with humans is inevitable: autonomous cars navigate through crowded cities, assistive robots help end-users with daily living tasks, and human engineers iteratively tune robot objective functions. In this graduate seminar class, we will build the mathematical foundations for modeling human-robot interaction, investigate algorithms for robot learning from human data, and develop the tools to analyze the safety and reliability of robots deployed around people. The approaches covered will draw upon a variety of tools such as optimal control, dynamic game theory, Bayesian inference, and modern machine learning. Throughout the class, there will also be several guest lectures from experts in the field. Students will practice essential research skills including reviewing papers, debating, writing project proposals, and technical communication.

16-887 Special Topic: Robotic Caregivers and Intelligent Physical Collaboration: Spring: 12 units
Robotics researchers and futurists have long dreamed of robots that can serve as caregivers. In this project-based course, you'll learn about intelligent physical human-robot collaboration and opportunities for robots that contribute to caregiving. You'll gain hands-on experience with teleoperation, autonomy, perception, navigation, manipulation, human-robot interaction, and machine learning. You'll also learn about robot design, collaborative research, and healthcare robotics.

Course Website: https://zackory.com/rc2023/

16-888 Special Topic: Foldable Robots: Origami-inspired design meets mechatronics: Intermittent: 12 units
The way we make robots have changed dramatically since the limitations on the material space was removed. Instead of using "nuts-and-bolts" approach that helped us to make robust, rigid, industrial robots, we can make light-weight, compliant, conformable robots out of paper, fabric, and polymers. In this class, we will explore foldable robots with a multifaceted perspective: Kinematics, design, fabrication, control, and application. We will design and manufacture mechanisms for targeted applications, such as manipulation, bio-inspiration, medical, architecture, using laminates with integrated joints and limited number of actuators.

16-889 Special Topic: Learning for 3D Vision: Spring: 12 units
Any autonomous agent we develop must perceive and act in a 3D world. The ability to infer, model, and utilize 3D representations is therefore of central importance in AI, with applications ranging from robotic manipulation and self-driving to virtual reality and image manipulation. While 3D understanding has been a longstanding goal in computer vision, it has witnessed several impressive advances due to the rapid recent progress in (deep) learning techniques e.g. differentiable rendering, single-view 3D prediction. The goal of this course is to explore this confluence of 3D Vision and Learning-based Methods.

16-890 Special Topic: Robot Cognition for Manipulation: Intermittent: 12 units
This seminar course will cover a mixture of modern and classical methods for robot cognition. We will review papers related to task planning and control using both symbolic and numeric methods. The goal of this course is to give students an overview of the current state of research on robot cognition.

16-891 Multi-Robot Planning and Coordination: Spring: 12 units
The course provides a graduate-level introduction to the field of multi-robot planning and coordination from both AI and robotics perspectives. Topics for the course include multi-robot cooperative task planning, multi-robot path/motion planning, learning for coordination, coordinating robots under uncertainty, etc. The course will particularly focus on state-of-the-art Multi-Agent Path Finding (MAPF) algorithms that can coordinate hundreds of robots with rigorous theoretical guarantees. Current applications for these technologies will be highlighted, such as mobile robot coordination for warehouses, drone swarm control, and multi-arm assembly. The course includes lectures, research paper presentations and discussions, and course projects.

16-892 Seminar: Multimodal Foundational Models: Fall: 12 units
This course will discuss recent foundation models proposed in the literature, with a focus on vision-language models. Topics include large language models, vision-language models, and vision-audio models. As time allows, this course will also discuss application of such models to visual, audio, and video content generation.
Prerequisite: 16-820

16-895 Understanding and Critiquing Generative Computer Vision: Spring: 12 units
In recent years, there have been significant advances in the field of large-scale generative modeling for visual data, such as DALL·E 2 and Stable Diffusion. This seminar course explores these advances beyond just reading and discussion. The goal is to not only inform state of the art but also develop critical and philosophical thinking among students. The course will involve reading papers, presentations, and discussions. The course will also involve reviewing and developing critical thinking.

16-901 RI-JEDI: Intro to Justice, Equity, Diversity, and Inclusion in Robotics: Fall and Spring: 3 units
This course will be offered in the first six weeks of the Fall semester, and will cover topics related to diversity, equity, and inclusion. This will be a companion course to CS-JEDI: 15-996. The course will be discussion-based and feature guest speakers. This course is offered to both graduate and undergraduate students in the RI. Designed specifically with the needs of PhD students in mind, the course is short, flexible, literature-based, framed through the lens of robotics and computer science, and is geared towards building community. We expect the students to spend and lt; 3 hours per week, almost all self-contained within the class time.

Undergraduate Catalog

Robotics Program

B.S. in Robotics Curriculum

Robotics core

ETHICS COURSE

mathematics

Robotics Electives

School of Computer Science Electives

SCIENCE AND ENGINEERING

HUMANITIES AND ARTS

FREE ELECTIVES

Double Counting

UNDERGRADUATE RESEARCH THESIS

Summary of Degree Requirements

Sample Course Sequence

Minimum number of units required for the degree:360

Robotics Additional Major

Required Courses

Double-Counting Restriction

Robotics Minor

Admission

Prerequisite

Required Courses

Electives

Double-Counting Restriction

Robotics Courses

About Course Numbers:

Print Options