Demo project: AI customer support agent connected to ticketing

This demo project shows how we build a level 1 AI support agent: it is not a client deployment but a complete system we maintain to illustrate our architecture choices and guardrails.

The modelled case is a software vendor whose support team receives repetitive requests (configuration, billing, common errors) whose answers already exist in the documentation and help articles.

The agent answers simple requests relying exclusively on that documentation, creates or updates tickets in the ticketing tool, and hands off to a human agent as soon as a request falls outside its scope.

An assistant based on a generic language model makes up answers, does not know the product and has no idea of the customer's context. Without guardrails, it creates more work for support than it removes.

Answers therefore have to be grounded in the documentation, what the agent is allowed to do has to be bounded, answer quality has to be measured and model call costs have to be controlled.

A Python service (FastAPI) receives messages from the chat widget or from incoming tickets in the ticketing tool via webhook.

An ingestion pipeline splits the documentation (help pages, release notes, FAQ) into chunks, computes their vector embeddings and stores them in PostgreSQL with the pgvector extension, along with version and source metadata.

For every question, the service retrieves the most relevant chunks (vector search combined with keyword search), builds a constrained prompt and calls the language model API.

A guardrail layer validates the input (sensitive data detection, off-topic requests) and the output (presence of sources, absence of forbidden commitments) before anything is sent to the user.

The agent has a limited set of tools: read the ticket, add a reply or an internal note, change its status and hand it off to a human. Redis manages conversation state and cost caps.

Every call is traced (OpenTelemetry) with token count, latency and cost, and an evaluation set is replayed automatically in the continuous integration pipeline.

The core of the approach is retrieval-augmented generation (RAG): the agent only answers from the retrieved documentation chunks and cites the corresponding pages. If the search returns nothing relevant or confidence is low, the agent says so clearly and offers a handoff rather than guessing.

Guardrails are implemented as separate steps, testable independently of the model: filtering of out-of-scope requests, masking of personal data in logs, an explicit list of allowed actions and verification of the answer before sending. An answer that fails these checks is replaced by a handoff.

Human handoff is a normal case, not a failure: the agent summarises the request, attaches the conversation and the sources consulted to the ticket, and places it in the right queue. The human agent sees exactly what was said and why the handoff happened.

Quality and costs are measured continuously. An evaluation set of representative questions with expected answers is replayed on every change; regressions block the deployment. Costs per conversation are tracked and a cap suspends the agent in favour of humans if the budget is exceeded.