Enterprise AI · Fertilizer & Clean Ammonia (Enterprise)
CFO Board Agent for Fertiglobe
Board-level Copilot Studio agent distilling source content into concise 3-line briefs on secure document grounding, at 87% evaluation accuracy.
- Role
- AI Engineer
- Designed the agent, document routing strategy, grounding and citation rules, and the response contract.
- Industry
- Fertilizer & Clean Ammonia (Enterprise)
- Timeline
- Delivered - v2
Business impact
- evaluation accuracy
- 87%
- brief format
- 3-line executive brief
- grounding
- Secure document grounding
- audience
- CEO & Board
Overview
Board-level Copilot Studio agent that distils source content into concise 3-line briefs from clear structured instructions, built on secure document grounding with a dedicated evaluation loop.
Structured instruction engineering plus a dedicated evaluation loop brought the agent to 87% evaluation accuracy while keeping every brief traceable to its grounded source.
Technical architecture
- Conversational Surface
- Copilot Studio agent with curated executive prompt starters
- Knowledge
- Nine curated document classes spanning financial, operational, strategic and market material
- Routing
- Question type to document class mapping applied before retrieval, falling through to the full corpus
- Retrieval
- Two attempt search discipline with synonym expansion on the second pass
- Grounding
- No general knowledge, no external sources, no recalculation, mandatory citation
Key features
- Closed corpus grounding - the agent answers only from curated official documents, never from training data
- Document routing before retrieval so the right source class is searched first
- Two attempt search rule before the agent may declare information unavailable
- Anti fallback rule - if the search returned content, the agent must answer from it
- Figures presented exactly as sourced, never recalculated or derived
- Mandatory data labelling - every figure states value, label and period
- Actual versus forecast discipline so a target is never presented as a result
- Conflict surfacing when documents disagree, with both values shown and flagged
- Mandatory citation of source document, section and date on every response
- Executive response contract - one sentence answer, supporting bullets, citation
Challenges solved
- Every answer influences an executive decision, so derivation is forbidden entirely by design
- Retrieval failure disguised as an answer - fixed with an anti fallback rule above every other instruction
- A heterogeneous corpus where the same question is answered well by one document class and badly by another
- Conflicting numbers across reporting packs surfaced rather than silently resolved
- Constraining formatting so an executive answer stays readable and actionable
Tech stack
- Microsoft Copilot Studio
- Azure AI Foundry
- Azure AI Search
- Azure OpenAI
- RAG
- Structured Instruction Engineering
- Knowledge Curation