# Securing AI/ML Systems: Protecting the Intelligent Attack Surface Canonical URL: ## Overview This is a practitioner course on securing AI and machine learning systems as systems, covering the data, the model, the pipeline and the application rather than treating AI as a policy topic. It opens by separating traditional cybersecurity risk from genuinely AI-specific risk, then runs the AI lifecycle from design through retirement with security gates at each stage. Adversarial machine learning is the core: poisoning attacks against training data, evasion attacks against deployed models, privacy attacks that target training information, and model extraction, followed by the defensive concepts that raise robustness, all grounded in the NIST adversarial machine learning taxonomy, AI 100-2 E2025. Generative AI gets its own module rather than a passing mention, covering direct and indirect prompt injection, sensitive information disclosure, insecure output handling and downstream application risk, and the risks introduced by retrieval-augmented generation, plugins, tools and external data sources. Two further modules cover ground that is usually missing entirely. The first treats trained models, weights and checkpoints as protected assets, with provenance verification and change management. The second addresses the AI supply chain — third-party models, datasets, libraries and frameworks as dependencies that can be malicious or vulnerable — because organizations routinely pull models and weights from public repositories with far less scrutiny than they would apply to a software dependency. The course closes with a live workshop in which participants threat-model a fictional AI system and defend a prioritized hardening plan. This is the engineering course rather than the policy course, written for the people securing the system rather than the people writing the policy; for the awareness tier, see [AI Security Fundamentals for the Cyber Workforce](https://www.nobledesktop.com/classes/ai-security-fundamentals-for-the-cyber-workforce). ## What you'll learn - Identify the components of an AI/ML system architecture and distinguish models, datasets, pipelines, applications and infrastructure - Distinguish traditional cybersecurity risk from AI-specific security risk - Map security considerations across the full AI lifecycle from design through retirement - Protect the confidentiality, integrity, availability and provenance of training and operational data - Identify data poisoning and manipulation risks across collection, labeling and preprocessing - Explain poisoning, evasion, privacy and model-extraction attacks, and apply defenses that increase robustness - Treat models, weights and checkpoints as protected assets, and verify provenance and integrity before deployment - Evaluate third-party models, datasets, libraries and frameworks as supply-chain dependencies - Identify direct and indirect prompt injection, insecure output handling, and retrieval-augmented generation risk - Secure ML pipelines, orchestration platforms, APIs and deployment processes - Detect abnormal inputs, outputs and model behavior, and distinguish performance degradation from compromise - Develop AI-specific incident response including containment, model rollback and post-incident validation ## Prerequisites Foundational knowledge of AI and machine learning systems. ## Curriculum #### Module 1 - Know What You're Defending: AI/ML Security Foundations - AI, machine learning, deep learning, generative AI and foundation-model concepts - System architecture components - Assets, trust boundaries, interfaces and dependencies - Traditional versus AI-specific risk - Security, resilience, robustness, privacy and trustworthiness #### Module 2 - Follow the Model: The AI/ML Security Lifecycle - Security across design, development, training, testing, deployment, operation and retirement - Responsibilities across developers, operators, users and service providers - Risks in experimentation and model development environments - Protecting development, testing, staging and production - Security gates through the lifecycle #### Module 3 - Data Is the Fuel: Securing Training and Operational Data - Training, validation, testing, inference and operational datasets - Confidentiality, integrity, availability and provenance - Access control for sensitive datasets - Data poisoning and manipulation - Protecting collection, labeling, transformation and preprocessing - Monitoring quality, integrity and lineage #### Module 4 - Attack the Learning Process: Adversarial Machine Learning - Attacker objectives - Poisoning attacks against training data and learning processes - Evasion attacks against deployed models - Privacy attacks targeting models and training information - Model extraction and information disclosure - Defensive concepts for robustness and resilience #### Module 5 - Guard the Model: Model Integrity & Intellectual Property - Models, weights, parameters and configurations as protected assets - Securing repositories and storage - Access control for model files, checkpoints and artifacts - Provenance and integrity verification before deployment - Theft, substitution and tampering - Secure versioning, approval and change management #### Module 6 - Secure the AI Supply Chain: Models, Libraries & Dependencies - Third-party models, datasets, libraries, frameworks and services as dependencies - Evaluating externally sourced components - Verifying provenance and integrity of acquired artifacts - Malicious or vulnerable dependencies in development environments - Controlling imports, updates, plugins and integrations - Ongoing supplier monitoring #### Module 7 - When AI Starts Talking: Generative AI & LLM Security - Security implications of large language models - Direct and indirect prompt injection - Sensitive information disclosure and unintended exposure - Insecure output handling and downstream risk - Retrieval-augmented generation, plugins, tools and external data sources - Isolation, access control, validation and least privilege for generative AI applications #### Module 8 - Secure the Pipeline: MLOps, APIs & Deployment - Protecting pipelines, orchestration platforms and automation workflows - Securing repositories, build environments and deployment - Protecting secrets, keys, tokens and credentials - Authentication and authorization for AI services and APIs - Separating development, training, testing and production privileges - Detecting unauthorized change #### Module 9 - Watch the Machine: Monitoring, Detection & Incident Response - Security logging and telemetry for AI environments - Monitoring model access, administrative actions, API activity and configuration change - Detecting abnormal inputs, outputs, behavior and usage - Distinguishing performance degradation from compromise - AI-specific incident response - Containment, model rollback, recovery and post-incident validation #### Module 10 - Govern, Test, Defend: AI Security Risk Workshop - Identify assets, trust boundaries and dependencies in a fictional AI system, develop AI-specific threat scenarios and attack paths, find data, model, pipeline, infrastructure and generative AI vulnerabilities, evaluate likelihood and impact, then select, prioritize and defend a risk-based hardening plan ## Schedule - Jan 5, 2027 – Jan 6, 2027 — Live Online - Feb 3, 2027 – Feb 4, 2027 — Live Online - Mar 22, 2027 – Mar 23, 2027 — Live Online - Apr 6, 2027 – Apr 7, 2027 — Live Online - May 4, 2027 – May 5, 2027 — Live Online - Jun 23, 2027 – Jun 24, 2027 — Live Online - Jul 6, 2027 – Jul 7, 2027 — Live Online - Aug 2, 2027 – Aug 3, 2027 — Live Online ## Pricing **Tuition:** $1049