Governed AI in Moodle: a buyer’s guide.

Adding AI to a learning platform is easy. Adding it in a way you can defend to your security team, your finance team, and your learners is not. This guide covers what "governed AI" should mean, the questions to ask any vendor, and — honestly — how we approach it and where the limits are.

Why "just add AI" is risky

Ungoverned AI in an LMS creates four practical problems:

• Data exposure — prompts and content leaving your control or training external models
• Cost runaway — usage with no caps producing an unpredictable bill
• Wrong answers presented confidently — unhelpful for teaching and hard to audit
• Shadow AI — staff pasting content into consumer tools because the platform offers nothing governed

What governance should actually include

• Audit — a record of AI actions you can review
• Spend controls — per-tenant caps and rate limits, with a defined behaviour when a cap is reached
• Data handling — clarity on where prompts and content go, retention, and whether anything trains external models
• Policy and access — which roles can use which features, and content-safety controls
• Human review — people approve AI-generated material before it reaches learners
• Provider control — knowing which model provider is used and in which region

Questions to ask any vendor

• Where do our prompts and content go, and are they used to train anyone's models?
• Can we set a spend cap per tenant, and what happens when it is hit?
• Is there an audit trail of AI actions, and who can see it?
• How do you reduce the chance of the assistant answering outside approved content?
• Which model provider and region process our data?
• Who reviews AI-generated content before students see it?

How ScaleMoodleLMS approaches it

Our AI runs behind a governance layer rather than being wired straight into Moodle:

• The assistant is designed to answer from approved learning content, provide citations, and refuse when sufficient supporting information is not available
• Per-tenant spend caps and rate limits, with a fail-closed behaviour when a cap is reached
• An audit trail of AI and governance actions for review
• Best-effort PII redaction on prompts before they reach the model provider
• Processing via Amazon Bedrock, with in-region options for dedicated deployments
• Human-in-the-loop: teachers and admins approve AI-drafted courses, quizzes and pathways before publishing

We are deliberate about what we do not claim. AI systems can still be wrong, and redaction is best-effort, not a guarantee. Governance reduces and bounds these risks and keeps a human in control — it does not make them disappear.

Red flags to watch for

Be cautious of absolute claims — "it cannot hallucinate", "it only ever tells the truth", "it can never leave the syllabus". No current AI system can guarantee that. A credible vendor describes controls and limits, not magic.