Practical AI for Existing Business Systems: RAG, Tools and Controlled Automation

History and context

Keep deterministic logic deterministic

AI is useful where language interpretation and uncertainty are part of the problem. It is a weak replacement for rules that can be expressed exactly.

RAG grounds answers in company knowledge

A bounded .NET orchestration pattern

public async Task<AssistantAnswer> AskAsync(
    string question,
    CancellationToken cancellationToken)
{
    var sources = await search.SearchAsync(
        question,
        top: 5,
        cancellationToken);

    var context = string.Join(
        "\n\n",
        sources.Select(x => x.Content));

    var answer = await model.GenerateAsync(
        BuildPrompt(question, context),
        cancellationToken);

    return new AssistantAnswer(answer, sources);
}

Use approved tools, not arbitrary production access

Do not connect a model directly to unrestricted SQL. Expose bounded functions whose validation and authorisation are implemented in normal application code.

public interface IOrderTool
{
    Task<OrderSummary> GetOrderAsync(
        long orderId,
        CancellationToken cancellationToken);
}

Support copilot example

A support assistant can combine order data, customer history and policy search, then draft a response. The model does not invent the order state; it receives it from the authoritative API.

Permissions and prompt injection

AI retrieval must inherit tenant, role and document-access boundaries. Retrieved content should also be treated as untrusted input because documents and external text can contain instructions that attempt to manipulate the model.

Evaluation before production

A production AI feature needs a realistic evaluation set covering grounding, source relevance, refusal when evidence is insufficient, permission enforcement, latency and cost. A good demo is not a production test.

Question:
Can a customer return an opened item after 45 days?

Expected evidence:
Returns Policy, section 4.2

Expected behaviour:
Do not claim the return is allowed
if the approved policy limits returns to 30 days.

A current Azure-oriented RAG shape

Microsoft's current Foundry guidance describes RAG patterns that use enterprise knowledge retrieval and can use Azure AI Search or Foundry knowledge capabilities to ground model responses. The specific service choice should follow the workload and governance requirements.

Business value and practical considerations

References