Natural Language Programming

Loops

JoeQ Framework

Partial Redundancy Elimination

Dataflow Analysis

Constant Propagation

Register Allocation

Instruction Scheduling

Software Pipelining

Parallelization

Loop Transformations

Pipelined Parallelism

Pointer Analysis

BDDs in Pointer Analysis

Satisfiability Modulo Theories (SMT)

Garbage Collection

CS 243: Program Analysis and Optimizations Stanford University Winter 2023 Focuses on efficient high-level programming through code optimization and program analysis for quality improvement. Also explores automatic memory management and natural language coding through machine learning. ## What do we teach?

- How do we make high-level programming languages efficient by optimizing the code? (Data-flow analysis)
- Machine learning and scientific computing need a lot of cycles; machines such as GPUs are complex to code. How do we automatically generate efficient code for these machines effectively. (Parallelism and Locality)
- Can we use program analysis to improve the quality of the code such as detecting security bugs in programs? (Pointer analysis, Satisfiability Modulo Theories)
- How do we automatically manage memory efficiently so users do not have to manage it themselves? (Garbage collection)
- The highest programming language is obviously natural language. With the recent introduction of large language models such as GPT3 and CodeX, how do we let consumers code in natural language using machine learning? Note: no prior knowledge in machine learning is needed. (Neural networks)

## How do you learn this?

As compilers are one of the most complex programs we write, by teaching compilers, we are also teaching software engineering. It is a course where programming and mathematics meet. We show how to apply general mathematical concepts (fixpoint computations, graph theory, linear algebra, binary decision diagrams, neural networks) to handle various complex programming problems. You will learn how to formulate solutions to problems, and not just memorizing prior work. ## Course Emphasis

- **Methodology: apply the methodology to other real life problems**
    - Problem statement
        - Which problem to solve?
    - Theory and Algorithm 
        -Theoretical frameworks 
        - Algorithms
    - Experimentation: Hands-on experience (Weekly programming/written homeworks)
- **Compiler knowledge:**
    - Non-goal: how to build a complete optimizing compiler
    - Important algorithms
    - Exposure to new ideas
    - Background to learn existing techniques CS 103 or CS 103B, and CS 107; Java programming language experience 

CS 243: Program Analysis and Optimizations

Compilers

Natural Language Processing (NLP)

Task-Oriented Agents

Knowledge Base Query Agents

GenieScript

Architecture of Virtual Assistants

Multimodal Virtual Assistants

Training Conversational Virtual Assistants

Automatic Speech Recognition

Multilingual Virtual Assistants

Text-based Question Answering

General Virtual Assistants

Large Language Model (LLM)

CS 224V Conversational Virtual Assistants with Deep Learning Stanford University Fall 2022 This course focuses on the creation of effective, personalized, conversational assistants using large language neural models. It involves both theory and practical assignments, offering students a chance to design their own open-ended course project. Familiarity with NLP and task-oriented agents is beneficial. Computers will transform into effective, personalized, conversational assistants for everybody, including the pre-literate and the non-literate. Commercial chatbots today are notoriously brittle as they are hardcoded to handle a few possible choices of user inputs. Recently introduced large language neural models, such as GPT-3, are remarkably fluent, but they are often erroneous and prone to hallucinations. This course studies how we can tame these neural models into robust, trustworthy, and cost-effective conversational agents across all industries and languages.

Students will learn both the theory and practice with **two assignments**, and a **major open-ended course project** of their own design.

Topics include:

- Formal, executable meaning representation for conversational virtual assistants
- Contextual neural semantic parser for translating dialogues into a formal representation.
- Automatic generation of training data for dialogues from high-level schema and API specification with large language models.
- Using large language models in virtual assistants.
- Multilingual, mixed-initiative, multimodal assistants.
- Federated privacy-protecting assistants.   

CS 224V Conversational Virtual Assistants with Deep Learning

Deep Learning