Description
Self-Hosting LLMs Securely: Hardening Ollama and vLLM. This advanced security training course teaches administrators and engineers how to securely deploy and protect large open source language models on their own hardware. This course provides a structured approach to securely running and protecting open source AI models on their own infrastructure. Running a model on dedicated hardware preserves the privacy of data and requests, but also puts the user in charge of a new server. In this training, a model server is treated as a real server; the default settings are first checked, then layer-by-layer hardening is applied, and each change is verified with automated verification scripts. Learners begin by evaluating and mapping the Ollama and vPython APIs to identify default vulnerabilities. Then, Olama is hardened on the host system by setting connection addresses, restricting access, using a dedicated service user, and the SystemD sandbox. In the next step, the same service is run inside secure containers with limited access and approved images. The Vpython section includes a thorough review of security recommendations, flag settings, and gateway configuration with security certificates, private keys for each client, request rate limiting, and a proxy layer. The host is also isolated through firewalls, and model files are carefully scanned, signed, and checked before execution. In the final sections, security monitoring with tools such as Prometheus, Grafana, and Falco, secrets management, security incident simulation, and compliance with globally recognized standards such as INIST and ISO standards are reviewed to achieve complete defense skills.
What you will learn
- Drawing a threat model of a self-hosted inference server including assets, trust boundaries, and default vulnerabilities
- Measuring and assessing the default state of Ulama and Vpython before applying changes and recording baseline scores
- Locking Ulama on Lookpack, restricting Keras access, running it with a dedicated user, and isolating it with SystemD
- Running Ulama and Vpython inside secure containers as non-root, read-only, and published on Lookpack
- Correct use of Python-specific flags and closing unprotected endpoints
- Setting up an authentication gateway against the model with support for security protocol, client keys, and rate limiting
- Blocking incoming and outgoing traffic with firewall tools and keeping the model server away from the public internet
- Validation of model files before serving, including format checks, security scans, signatures, and model manifests
- Event logging, monitoring, and alerting on the model host with Prometheus, Grafana, and Falco, and detecting unauthorized changes
- Regular monitoring, conducting incident response exercises, and preparing documentation of compliance with security standards
This course is suitable for people who:
- Infrastructure, DevOps, and security engineers who are tasked with implementing open source models on their organizational infrastructures
- Security teams are responsible for reviewing and auditing internal deployments of Ulama or Vpython before entering the operational environment.
- Personal lab operators and small teams who want a fully secure, local model server
Course Details Self-Hosting LLMs Securely: Hardening Ollama and vLLM
- Publisher: Udemy
- Instructor: NEXUS ACADEMY
- Training level: Beginner to advanced
- Training duration: 10 hours and 33 minutes
- Number of lessons: 109
Course topics
Prerequisites for Self-Hosting LLMs Securely: Hardening Ollama and vLLM
- Comfort with the Linux shell, systemd and Docker basics. No machine-learning background is needed
- A Linux machine or VM that can run Docker. No GPU is needed: every lab runs on CPU with small, openly licensed models
Course images
Sample course video
Installation Guide
After Extract, view with your favorite player.
Subtitles: None
Quality: 720p
Download link
Rapidgator link
File(s) password: www.downloadly.ir
File size
1.7 GB