Master’s in Model and Data Observability Engineering
Online master's programme of 1,900 hours in model and data observability: 12 modules covering data observability, drift statistics, model monitoring, LLM observability, telemetry and instrumentation, explainability in production, alerting and incident response, retraining loops and AI governance, with an assigned technical tutor.
Take the controls.
Run a real engineering trade-off from this field. Move the sliders, watch the model react in real time and beat the challenge.
What you will be able to do
Design observability strategies for data and models.
Detect data quality anomalies in pipelines.
Measure data and concept drift with statistical methods.
Instrument AI services with OpenTelemetry.
Evaluate and monitor LLM-based systems.
Manage alerts and model degradation incidents.
Meet the monitoring requirements of the European AI Act.
Who it's for
- ML observability engineers responsible for models running in production.
- Data reliability engineers who keep critical pipelines healthy.
- MLOps engineers adding monitoring and retraining to their deployments.
- SREs working on AI platforms and inference services.
- AI systems auditors who need traceability and evidence of oversight.
- Data quality leads setting up tests, monitors and data contracts.
- Engineering or science graduates aiming to move into banking and insurance.
What you'll work with
- Regulation (EU) 2024/1689 (AI Act) (European Artificial Intelligence Regulation)
- ISO/IEC 42001 (artificial intelligence management systems)
- NIST AI RMF 1.0 (NIST AI Risk Management Framework)
- ISO/IEC 23894 (AI risk management)
- ISO/IEC 25012 (data quality model)
- ISO/IEC 5259-1 (data quality for analytics and machine learning)
- Regulation (EU) 2016/679 (GDPR) (EU General Data Protection Regulation)
- Evidently
- OpenTelemetry
- Prometheus and Grafana
- Great Expectations
- Langfuse
12 modules · 1900 h
1.1Monitoring versus observability1.2The pillars: metrics, logs, traces and events1.3Silent failures in data and models1.4The cost of data incidents1.5The model lifecycle and points of failure1.6Roles: data engineering, ML and SRE1.7Observability maturity1.8The tooling landscape1.9Building the business case1.10Case study: analysis of a real data incident
2.1Freshness, volume, schema, distribution and lineage2.2Declarative data tests2.3Great Expectations and Soda2.4Testing in dbt2.5Anomaly detection in table metrics2.6Schema changes and contracts2.7Monitoring streaming pipelines2.8Column-level lineage with OpenLineage2.9Prioritising alerts by impact2.10Lab: quality monitors with Soda
3.1Reference and production distributions3.2The Kolmogorov-Smirnov test3.3Chi-squared for categorical variables3.4The population stability index3.5Kullback-Leibler and Jensen-Shannon divergences3.6Wasserstein distance3.7Multivariate drift and domain classifiers3.8Window size and statistical power3.9Correction for multiple comparisons3.10Practical: comparing drift metrics
4.1Covariate, label and concept drift4.2Performance with delayed labels4.3Performance estimation without labels4.4Probability calibration4.5Segment-level monitoring4.6Fairness metrics in production4.7Monitoring time-series models4.8Monitoring computer vision4.9Embeddings and drift in unstructured data4.10Lab: monitoring with Evidently
5.1Tracing chains and agents5.2Cost, latency and token consumption5.3Reference-based response evaluation5.4LLM-as-a-judge evaluation5.5Hallucinations and grounding in RAG5.6Retrieval quality5.7Security: prompt injection and leaks5.8User feedback5.9Regression evaluation sets5.10Workshop: tracing a RAG application with Langfuse
6.1OpenTelemetry: API, SDK and collector6.2Metrics with Prometheus6.3Structured logs and their correlation6.4Distributed tracing of inference requests6.5Instrumenting inference services6.6Telemetry cardinality and cost6.7Trace sampling6.8Correlation between signals6.9GPU telemetry6.10Practical: instrumenting an inference service with OpenTelemetry
7.1Global and local feature importance7.2SHAP in monitoring7.3Attribution drift7.4Counterfactuals and recourse7.5Explanations for auditors7.6Explainability of deep models7.7Limitations of explanations7.8Logging explanations per prediction7.9Communicating with business users7.10Lab: SHAP attribution drift
8.1Designing actionable alerts8.2Static versus dynamic thresholds8.3Alert fatigue8.4SLOs and error budgets for ML8.5Runbooks for model degradation8.6On-call management8.7Root cause analysis with lineage8.8Rolling back models and data8.9Blameless postmortems8.10Case study: drift incident drill
9.1Reference architecture9.2Metrics and log stores9.3Grafana and dashboards9.4Commercial ML observability tools9.5Open-source tools9.6Integration with the model registry9.7Integration with orchestrators9.8Multi-tenancy and access control9.9Cost and retention9.10Workshop: observability platform with Grafana and Prometheus
10.1Retraining triggers10.2Scheduled versus event-driven retraining10.3Validating candidate models10.4Shadow and canary deployments10.5Managing evaluation sets10.6Active learning and labelling10.7Harmful feedback loops10.8Data and experiment versioning10.9Safe automation of the retraining cycle10.10Practical: drift-triggered retraining pipeline
11.1Post-market monitoring under the AI Act11.2Event logging and traceability11.3AI management according to ISO/IEC 4200111.4Risk management according to the NIST AI RMF11.5Model and data documentation11.6Internal and external audits11.7Human oversight of systems in production11.8Reporting serious incidents11.9Regulated sectors: banking and healthcare11.10Case study: monitoring a high-risk system
12.1Selecting the data or AI system12.2Failure risk analysis12.3Designing the observability strategy12.4Instrumentation and telemetry12.5Data and model monitors12.6Alerting and response12.7Evaluation with simulated incidents12.8Conclusions and operational lessons12.9Thesis report12.10Master's Thesis: defence before a panel
How you'll learn
100% online training on the SEIUM campus, with downloadable materials, recorded classes and an assigned technical tutor. There are no fixed timetables: you progress at your own pace within each intake, so the programme fits around your job.
Each module combines theory, guided reading of the original standard and applied exercises. The working software is Evidently, OpenTelemetry, Prometheus and Grafana, Great Expectations and Langfuse, in free, academic or evaluation versions. Real procedures are studied, such as drift monitoring in credit risk models, observability of critical data pipelines and continuous evaluation of a RAG application. Assessment is continuous by module, plus the final Master's Thesis.
Standards covered: Regulation (EU) 2024/1689 (AI Act), ISO/IEC 42001, NIST AI RMF 1.0, ISO/IEC 23894, ISO/IEC 25012, ISO/IEC 5259-1, Regulation (EU) 2016/679 (GDPR).
Build it end to end
Your final project follows the real engineering workflow, step by step, reviewed by your tutor.
Where this takes you
Requirements
- University degree in engineering, science or architecture, or equivalent demonstrable professional experience.
- Internet connection and a computer capable of running the programme software.
- English sufficient to read technical standards.
Ready to start?
Send your application today. Our admissions team replies within 72 hours with your study plan and funding options.
Frequently asked questions
The programme totals 1,900 hours over 12–14 months. You set your own pace within each intake, and your tutor helps you plan the workload around your job.
You work with Evidently, OpenTelemetry, Prometheus and Grafana, Great Expectations and Langfuse, using free, academic or evaluation versions. Your tutor indicates the free alternative wherever one exists.
A SEIUM Institute diploma, awarded on passing the continuous assessment and the Master's Thesis. It is SEIUM's own qualification, not an official state degree; its recognition depends on the programme completed and the regulations of each country.
Yes. Modules passed in a course or postgraduate diploma in the same area are recognised in the higher programme. Ask about the recognition policy when you enrol.
A university degree in engineering, science or architecture (or equivalent professional experience), a computer able to run the software and English sufficient to read technical standards.
Start dates are set by SEIUM's faculty for each intake. Our admissions team will confirm the next available date when you apply.
Related programmes
Master’s in Time Series, Signals and Anomaly Detection Engineering
Online master's programme of 1,900 hours in time series, signals and anomaly detection: 12 modules covering the nature of anomalies, signal preprocessing, classical statistical methods, change…
Master’s in Traffic and Mobility Twins Engineering
Online master's programme of 1,900 hours in traffic and mobility digital twin engineering. 12 modules covering traffic engineering, traffic models, mobility data, real-time traffic twins, AI-based…
Master’s in Multimodality Engineering
Online master's programme of 1,900 hours in multimodal AI engineering. 12 modules covering modality fusion and alignment, per-modality representations, transformer architectures, contrastive learning, multimodal language models,…
More in Data, AI, Simulation & Analytics
All Data, AI, Simulation & Analytics programmes → · How to apply · Fees & financing · Scholarships