What ten years of production AI taught me

I have been putting AI into production since 2016, first with computer vision and language models, and now with retrieval-augmented LLMs. Across each generation, the hard part has been the data, integration and operating model around the model.

From 2016 I ran an innovation lab inside Société Générale, about twenty of us plus a rotating cast of startups. The word “AI” was in the air, but it meant something narrower than it does today: some computer vision, including optical character recognition, natural-language classifiers, and speech. We trained models on TensorFlow and PyTorch, later on AWS SageMaker, on a machine we built ourselves and kept in a locked room: a Threadripper, gaming GPUs, more RAM than anyone would sign off on twice. The big vendors were selling cognitive services by the API call, but a bank could not send its data out to them, so we built our own: OCR, speech to text, text to speech, language understanding, all open source, all on infrastructure we controlled, where the data stayed. That instinct, own the thing and keep the data in the house, came back years later.

One of those experiments became a chatbot we called YODA, Your Own Digital Assistant. It answered the questions employees used to send to a human and then wait on: IT, HR, compliance, the everyday friction of a bank with more than 150,000 people. We industrialised it to around twenty thousand users, in French and English, years before ChatGPT taught everyone the word “prompt.” It was not magic. It was intent classification, a retrieval layer, and a great deal of unglamorous work on the data behind it, built in the lab with Do You Dream Up as our conversational-AI partner. It took real load off the support teams, which was the only metric that mattered.

Around it we built the things that sound quaint now and were hard then: computer-vision pipelines, including OCR on Tesseract, speech models, even a couple of Pepper robots we deployed just to learn what physical presence added. Most of it worked. Some of it did not. All of it taught the same lesson.

The model was never the hard part

Every wave of AI arrives with the same story: the model is the breakthrough, and everything else is a detail. It is the reverse. The model is the easy part. It is a download, a paper, an API call. The hard part is everything around it.

It is the data, which is dirty, and which nobody owns cleanly. It is production, where the interesting failure modes live and where “it worked in the notebook” means nothing. It is trust, because a bank will not put a black box in front of a customer, and it should not. It is the operating cost, which is real and recurring and easy to wave away in a demo.

I spent those years learning to build the parts that are not the model. That is what made the difference every time the model changed.

The lesson kept being true

At Société Générale Equipment Finance I built a digital factory from nothing in Berlin and put an AI into production that read the documents behind a financing decision. The project had a name that gave away the goal, Need for Speed, and a more sober one for the slides, KYC 4.0. It used computer vision, including OCR, to pull the numbers out of balance sheets and the rest of the KYB and KYC paperwork, then machine learning, trained on Azure, to run the calculations and produce a risk score. It took a review that used to run to about two weeks and brought it down to roughly a day, in production, not on a slide.

The breakthrough was not the model. It was everything we built around it so the risk analysts would trust it. They reviewed every recommendation and could go against it. In one click they saw exactly where each figure came from: the source document, with every extracted number boxed and highlighted in place. No black box. The machine did the tedious extraction and the arithmetic, and the analyst kept the judgement. That is why it shipped, and why people actually used it. It started as a hackathon idea and made it all the way to production. Same lesson, four years later.

Building on a model we could operate ourselves

Fast forward to Chantelle. Generative AI had arrived, the models were extraordinary, and the temptation was to treat them as the answer to every question. We did not.

We put retrieval-augmented generation into production for the e-commerce and customer-service teams: a bot in Google Chat they could tag in a channel, orchestrated end to end in n8n, that answered their questions about our systems and processes so they stopped queuing for a person. It did more than answer. With an explicit confirmation from the person, it could take actions in their tools through MCP connectors, official ones and ones we built. The human stayed in the loop by design, the same instinct as the risk engine years before. For its first year it ran on Gemini. In June 2026 we shipped our own fine-tuned Gemma, self-hosted and served with vLLM, because owning the model meant owning the quality, the latency and the cost. Evaluation ran in CI, so a change that made answers worse failed the build. A feedback loop turned real failures into the next round of training data.

The most honest measure of whether it worked was not a dashboard. Early on, people still brought their questions to my team. I would ask, did you ask the bot? Yes, they would say, and it did not answer. I would paste the same question into it, and it would answer correctly. That gap, between what the tool could do and what people trusted it to do, is the real work, and it closed over time. The questions to my team dropped. That is adoption you cannot fake.

The model changed. The lesson did not. Retrieval quality beat prompt cleverness. Evaluation beat vibes. The unglamorous plumbing decided whether the thing was dependable or just impressive in a demo.

What this means if you are evaluating AI now

If you lead a team that is being asked to “do something with AI,” here is what ten years of it has taught me.

Start from a problem that costs you real money or real time, not from the technology. Measure the thing you actually care about, before and after. Assume the model is the cheapest part of the system and budget your effort accordingly: data, evaluation, and production are where the work is. Ship to a small group, learn, then widen. And be honest about cost, because an AI feature that quietly triples your inference bill is not a win.

None of that is new. I was applying it in 2016 with models that would embarrass a modern phone. The tools got better. The discipline is what carries over.

That is the approach I would bring to my next engineering leadership role: start with a real problem, build the operating system around the model and measure whether people use it.