Enterprise AI · Fertilizer & Clean Ammonia (Enterprise)
DOA: Delegation of Authority Agent for Fertiglobe
Copilot Studio policy Q&A over colour-coded DOA documents, where a preprocessing pipeline converts colour semantics to Markdown. 95% on 200+ eval cases.
- Role
- AI Engineer
- Owned agent design, instruction engineering, the knowledge pipeline and the evaluation loop.
- Industry
- Fertilizer & Clean Ammonia (Enterprise)
- Timeline
- Delivered - 18 instruction versions across two evaluation cycles
Business impact
- accuracy
- 95%
- evaluation cases
- 200+
- data fix
- Colour-coding to Markdown
- integration
- SharePoint + Entra ID
Overview
Policy Q&A via Copilot Studio over Delegation of Authority documents. The source PDFs encoded approval values as colour coding rather than literal numbers, so a preprocessing pipeline converts colour-coded documents into clean Markdown before the agent ever sees them, fixing the data problem ahead of the agent problem.
SharePoint and Entra ID integrated for grounded, permission-aware retrieval. Reached 95% accuracy across 200+ evaluation cases.
Technical architecture
- Conversational Surface
- Copilot Studio agent with generative orchestration and a conversation start disclaimer
- Knowledge
- Tiered governance documents prepared into structured markdown - matrices, limits, definitions, appendices
- Retrieval
- Azure AI Search knowledge sources with a mandated search order and attempt budgets
- Change Control
- A canonical behaviour document as single source of truth, with a fixed propagation order
- Evaluation
- Versioned evaluation datasets across two rounds driving eighteen instruction iterations
Key features
- Five way intent classification evaluated in strict order, each with its own response contract
- Mandatory search before answering with per tier keyword attempt budgets
- Tier priority resolution so board level answers win and identical outcomes are never duplicated
- Threshold scoping returning only the band containing the amount the user gave
- Currency normalization before threshold matching, with the conversion stated
- Single question clarification rule when the amount or activity is missing
- Strict one row data verification so endorser and approver never mix across rows or tiers
- Deterministic output contract with a reference documents footer on every grounded answer
Challenges solved
- Eighteen instruction versions had drifted - resolved by freezing one canonical behaviour document
- Preventing invented approvers with one row verification and an explicit not found response
- Over answering - the threshold scoping rule cut replies to the single applicable band
- Two overlapping governance tiers handled with explicit priority and de duplication rules
Tech stack
- Microsoft Copilot Studio
- Azure AI Foundry
- Azure AI Search
- SharePoint
- RAG
- Structured Instruction Engineering
- Evaluation Harness