Flexible data platforms can supercharge government GenAI adoption

Over the past 18 months, more people have searched for ChatGPT than Taylor Swift, according to Google search trends. When it comes to capturing the public imagination, it appears the famous singer has trouble shaking off the allure of generative AI (GenAI).

Of course, the excitement users have for experimenting with and using GenAI needs little explanation – the emerging tech is expected to boost productivity, simplify work, and support new forms of creativity.

This is true for the public sector as well: agencies around the world are already using GenAI to improve communications with citizens, speed up repetitive tasks, and support employees.

However, robust artificial intelligence (AI) requires access to robust data to ensure answers are validated – and governments have notoriously complex and vast data structures, which can make it difficult to drive GenAI adoption at scale.

This was one observation I shared at a recent roundtable hosted by GovInsider in collaboration with Redis, a real-time data platform company.

To subscribe to the GovInsider bulletin click here.

Why a flexible platform approach

During the event, I highlighted the value of a platform approach to enable GenAI innovation.

Governments face an uphill battle when adopting GenAI due to the complexity of existing data structures. In the public sector, there are vast amounts of data that are siloed and inaccessible, with varying confidentiality levels. Agencies need to find a way to use such data more efficiently to solve business needs.

In recent years, Redis has been widely adopted by government agencies across the globe as an efficient and flexible real-time data platform which can quickly retrieve data from a variety of legacy sources while remaining secure against data leakage.

This can enable agencies to improve customer experience and administrative efficiency while enabling data-driven policymaking.

Singapore’s national health tech agency, Synapxe, has recently launched HEALIX, or Health Empowerment thru Advanced Learning & Intelligent eXchange, a cloud-based analytics platform which enables seamless data sharing across the public healthcare ecosystem, said Synapxe’s Deputy Director (Data Analytics & AI), Yogesh Pathak in a fireside chat.

By introducing a common data exchange platform, the agency aims to enable users to rapidly develop analytics and AI models to meet healthcare use cases. Similarly, they have also introduced a common Secure GPT Platform, Tandem, to enable healthcare professionals to tap on GenAI on a secure platform.

The agency is employing a wide range of cybersecurity measures, such as data encryption, privilege access management, and cloud security controls to ensure data remains secure, shared Pathak.

To subscribe to the GovInsider bulletin click here.  

Leapfrogging the legacy barrier

A key benefit of such an approach is that agencies can connect across a variety of data sources, including legacy systems, with the help of Redis.

For example, the Redis platform can automatically connect to legacy data sources (e.g. Relational Database Management Systems, RDBMS) and synchronise data without the need to write code. As the roundtable participants highlighted, some public sector agencies will continue to rely on legacy systems that have reliably supported mission objectives over the years.

Next, the platform uses tools such as retrieval-augmented generation, semantic caching, and LLM memory to ensure that AI applications can swiftly draw on a wider range of data sources and provide more relevant responses.

These can address common AI challenges, such as the risk of hallucinations, lag in response time, and a lack of access to real-time information.

The platform can also scale up to accommodate varying workloads even as it provides fine-grained security controls. Founders at OpenAI, which launched ChatGPT in 2022, have credited Redis for the rapid scaling of ChatGPT.

Redis is the real-time data engine that can be deployed either on an agency’s on-premise environments, or on a protected cloud environments, or on a hybrid of both on-premise and cloud.

Agencies have full control of data in Redis, at the same time maximize the capabilities of GenAI without worrying about sensitive or proprietary information leakage.

Starting with the right design

During the roundtable discussion, participants spoke about the need to design AI applications that solve clear use-cases in small groups. Leaders will need to identify what their objectives are before designing the right AI model, shared participants.

They may also wish to consider the best ways to integrate data. For example, the designers may choose to tap on microservices or lightweight APIs to connect data sources to AI applications.

For certain use-cases, leaders may also wish to tap on edge computing and bring AI closer to data sources to improve security.

Edge computing can enable more real-time applications, the roundtable participants noted, such as in the case of medical devices that need to quickly alert healthcare professionals when patients experience a medical emergency.

Participants also highlighted that the public sector would move more slowly than the private sector when it comes to adopting GenAI as it is highly regulated, which is one reason why a platform approach might be preferable.

Shi Lei is a seasoned professional with 14 years of experience in software development and product architecture design. Over the past 10 years, he has specialized as a Redis developer, and for more than 7 years, he has focused on AI architecture and data science. As the Senior Solution Architect at Redis, Shi Lei assists top-tier customers across various domains in designing and building solutions for real-time data processing. Prior to joining Redis, he developed multiple award-winning AI solutions for leading banks and insurance companies, showcasing his expertise and innovative approach in the field.