Hall Research AI

Cohort 1

Summer 2024

Validating and testing RAG systems to manage generative AI risks in real-world use.

ASRC Feb 2025

ASCR Spring 2026

In coordination with ASRC Federal, HallResearch.ai offer a second AI Ethics Technical Training Course.

Leader in Responsible AI & Risk Mgmt

Prof. Patrick Hall

Principal Scientist, HallResearch.ai and CAIO, GW School of Business

ASRC AI Ethics Technical Training Course

Information System Security Engineer

Database Administrator

Software Developer AI

BD Data Scientist/BD Operations Analyst

Learn to validate and govern RAG-based generative AI with cutting-edge human-calibrated testing and NIST AI 600-1 guidance.

Patrick Hall

Principal Scientist at HallResearch.ai, GWU Faculty

Agus Sudjianto

Senior VP, H2o.ai and Former EVP & Head of Model Risk, Wells Fargo

Governing GenAI - From Prompts to Policy

Software Systems Architect

Legal Director

Model validator

Director, AI Division

Senior Data Scientist

Director - Model Risk

In today's rapidly evolving artificial intelligence landscape, engineers and risk managers play pivotal roles in ensuring innovation and accountability. Our course on explainable AI (XAI) equips professionals with essential skills to navigate this rapidly expanding complex terrain. Students will gain hands-on experience exploring compliance considerations derived from real-world applications across diverse sectors. By merging technical expertise with compliance insights, this course empowers individuals to drive impactful, transparent AI solutions in their respective domains, fostering innovation and integrity. This course explores XAI, covering essential background definitions and concepts, explainable feature engineering, the diverse ecosystem of XAI models, post-hoc explanation methods, and the latest developments in audit and transparency laws and regulations. Some featured topics in this course on&nbsp;explainable AI (XAI): • The&nbsp;fANOVA&nbsp;framework• From&nbsp;penalized GLM to EBM and GAMinet•&nbsp;Monotonic GBMs•&nbsp;Surrogate model approaches•&nbsp;Explainable feature engineering•&nbsp;LOCO, LOFO, and pertubation• SHAP: The good, bad, and ugly• Security and bias considerations Background information and definitions are sourced from the National Institute for Standards and Technology (NIST), the National Academies, and other authoritative sources. The instructor teaches technical subjects with Python examples and compliance lessons based on his real-world experience in consumer finance, employment, and other regulated applications of ML. (Note: The instructor is not an attorney.) This course is designed primarily for data scientists and ML engineers. Technical risk executives or risk managers may find this course helpful in providing an updated overview of newer ML approaches suited for high-stakes applications. This course may interest others too. Materials may help regulators or policy professionals gain insights into the current state of ML technologies that can be used to comply with laws, regulations, or standards. If you're coming to ML from physics, econometrics, or psychometrics, this course can help you learn how to blend newer ML techniques with established domain expertise and notions of validity or causality.

Learn to build accurate, transparent, and interpretable ML/AI models. 

Course syllabus

Live lectures followed by discussion session.

Monday, July 8, 2024

Live lectures and demos followed by discussion session.

Wednesday, July 10, 2024

Monday, July 15, 2024

Wednesday, July 17, 2024

Course Schedule

ML engineers and data scientists who want to learn about XAI.

Technical risk executives seeking an updated overview of newer ML approaches suited for high-risk applications.

Policy professionals interested in the current state of ML technologies that may be used to comply with regulations and standards.

Who is this course for

Learning Objectives

<ul><li>Ecosystem of cutting-edge explainable ML models</li><li>Essential compliance awareness for ML transparency</li><li>Emergent audit and transparency laws and regulations</li></ul>

Build accurate, transparent, and understandable ML models.

<ul><li>Best practices for post-hoc explanation</li><li>Explainable feature engineering</li><li>Testing explanation quality</li></ul>

Interpret and explain sophisticated ML algorithms, enabling them to uncover insights, identify biases, and understand complex decisions.

Be equipped to build safer and more trustworthy AI systems, tackle real-world problems confidently, and drive positive impact in various domains such as healthcare, finance, and human resources.

Build safer and more trustworthy AI systems.

Learn to build accurate, transparent, understandable ML models. Get real-world policy and compliance insights for high-risk applications.

Principal Scientist, HallResearch.ai                                               
 & Assistant Professor, George Washington University

Evals Are Almost Never All You Need

Principal Scientist, HallResearch.ai

Understand how task contamination, conflicts of interest, and lack of scientific measurement can compromise evals.

Common problems with benchmarks and evals

Learn about embedding-based approaches, red-teaming, and field testing for evaluating generative AI systems.

What to do instead of benchmarks and evals

Developers need benchmarks. They're an important tool for development. They're the wrong tool for real-world assessment.

When to use benchmarks and evals

GenAI Validation - Free Discussion and Q&A Session

VP at H2o.ai (ex-Wells Fargo, B of A, and Ford Analytics Leader)

Principal Scientist at HallResearch.ai (Teacher, Author, Inventor, Advisor)

We'll have some questions for each other, then we'll open up the discussion for audience Q&A.

Meet Agus and Patrick

No stupid questions here, and we'll do our best to answer your difficult questions too.

Bring you simplest or hardest questions

We're offering a course on generative AI validation, July 22 - 24, 2025. We want to hear what potential students think!

Learn directly from Hall Research AI

Hall Research AI

Courses

Explainable AI for Decision-Making Applications

Lightning Lessons

Evals Are Almost Never All You Need

GenAI Validation - Free Discussion and Q&A Session

Understand SHAP (SHapley Additive exPlanations)

Understand SHAP (SHapley Additive exPlanations)