'Test hard and then be brave': What Germany's trade agency learnt adding AI to search

"It was astonishing to me that even our staff actually used Google to find our own website information," says Germany Trade & Invest (GTAI)'s senior online marketing manager, Yvonne Richtsteig. 

GTAI is the international economic promotion agency of the Federal Republic of Germany that helps German companies expand abroad and helps foreign investors set up shop in Germany. Its content spans more than 30,000 articles covering over 150 countries.  

If GTAI's own staff couldn't find their way around the information, how could the businesses then rely on it for advice?  

A business user could ask about custom regulations, and you would get over 100 results.

Sometimes, you just want to quickly get a clear answer, adds GTAI's senior online marketing manager, Maike Neumann.  

But that is now in the past. 

Recently, GTAI added an artificial intelligence (AI) search layer to its website, letting users ask questions in natural language and get answers referenced to the original source.  

It also provides an option for users to connect directly to a GTAI officer.  

According to the team behind the AI implementation, the technology itself was almost beside the point. 

What mattered more was what had to happen first: breaking down walls between teams, keeping AI on a tight leash, and testing hard enough to trust it - and at some point, being brave enough to trust it. 

You can't fix search from one department 

The numbers around the traffic loss and change in consumer search behaviour made the case as to why GTAI decided to look towards AI to improve the searchability of its content. 

"With the right numbers [presented by the web analysts], it changed the way that leadership and

ourselves looked at the situation. It became clear that we needed to do something now and take a closer look at our search," says Neumann. 

What followed wasn't handed to only one team, but with the agency forming a cross-functional working group called "Level Up," pulling in editorial staff, developers, UX designers and online marketing to build the search layer together.  

While the online marketing team could technically make the search function work, says Richtsteig, it has no authority over the content itself and needed the editors to check whether the AI's answers were accurate.  

As AI touches content, infrastructure, design and most importantly, public trust, the team tasked to deliver an AI project has to reflect the multi-disciplinary approach from the  first meeting.  

Never let AI go looking outside your own walls 

The team started with how people already search: "They just check the first few results, they don't browse through all the pages," Richtsteig says, adding that GTAI wanted its own website to work the same way. 

This translates to giving a customer's question one good answer, rather than a long list of results to sift through. 

And giving one confident answer raises the stakes on that answer being right. This shaped the rule that GTAI settled on, which was that its AI would only answer with GTAI's own content and never the open internet or outside partners. 

In the public sector, allowing AI to draw on outside content risks "giving control out of your hands," she adds. 

The rule is only workable because the content behind it was already in shape. GTAI's editors were already tagging and classifying their published articles in their content management system, CoreMedia. 

AI's role, enabled by the conversational search service provided by its AI implementation partner, ]init[, was then to put that existing structure to use as a "matchmaker for question and answer," she says. 

The system also updates in real time, giving the AI access to new content when it's published, instead of waiting for periodic retraining, she notes. 

"Be brave and trust that you have great quality content. That's the foundation: The sources, the content and have rules in place like the do's and don'ts," she says. 

Don't launch until your own experts believe it  

Before GTAI's AI search tool went anywhere near the public, about 20 subject matter experts were given a very specific job to check the AI's answers about their own subject areas.  

"We asked the experts who wrote the content to check their own content," Neumann says, adding that this meant verifying the sources cited, the contacts listed and whether AI was hallucinating.  

People check harder when the answer is attached to their name, which was why GTAI asked the original authors to test it themselves. 

Richtsteig adds that the testers checking answers about their own published work wouldn't let even a single wrong word slip past.  

"If it's under their name, they feel responsible for it," she says, describing the overall process as "very intense." 

Every issue they found went onto a shared dashboard.  

"We created our own ticket system, where we had a huge board where every bug, or everything which was not working quite well, was on that board, and we checked the ticket and made the adjustments," Neumann says.  

After the rigorous testing process, GTAI first opted for a soft launch with a beta version rather than a full public rollout. "We played it very safe," Richtsteig says 

"Because we know we have a lot of responsibility here, we have a reputation, and we want to live up to it." 

Since then, testing hasn't stopped after the full rollout.
 

The team still checks in on the tool regularly and plans to repeat the full process this year when GTAI swaps its underlying language model, which is a routine technical upgrade, Neumann says. 

As highlighted by the team, the key takeaway was never about being certain that nothing can go wrong but keep testing and still go for it.