Over the past few months, something subtle but significant has been happening across several of our digital services in the Department for Education (DfE). We’ve seen a rise in impressions coming from AI‑mediated search, at the same time as visits to our actual pages have flattened or dipped. More people are now receiving answers through Google’s AI Overviews, Microsoft Copilot and other emerging assistants, without ever visiting the services we’ve designed for them. At first, this felt like progress; faster access to information is not something to resist. But as we looked closer, a more complicated picture emerged.
The new reality: answers without journeys
Public services are not just providers of information. They are structured journeys intended to help people make sense of systems they may never have encountered before. These journeys carry context, safeguards, clarifications and next steps that ensure people understand not just a single answer, but the path around it. AI assistants, however, don’t really understand journeys; they understand questions. They extract the answer they think is most relevant, remove the supporting detail around it, and present it as a self‑contained response. The risk is not usually that they are wrong, though that can happen, but that they are incomplete in ways that matter.
What happens when users don’t know what they don’t know?
This incompleteness is especially challenging for the kinds of users many of our services are built to support. Not everyone arrives with a clear question. A teenager leaving school may not know to search for “apprenticeships”, “T Levels” or “vocational pathways”. They just know they need to figure out what to do next. Parents concerned about their child’s online safety often don’t know the terminology of the problems they’re worried about. Care‑experienced young people may not know that certain entitlements or support options even exist. A well‑designed service helps reveal possibilities, not just respond to questions. It helps people articulate what they need (or need to do), especially when they don’t yet have the language for it. AI tools, by contrast, only answer what they’re asked. They meet users where they already are, which can limit discovery and reinforce gaps in understanding.
Should we change the way we work or return to our foundations?
As a design team, we’ve been asking ourselves whether this moment requires us to rethink how we work, or whether it simply calls for a renewed commitment to the principles that have guided digital government for more than a decade. We keep coming back to the same idea: the fundamentals of user‑centred design still hold. If anything, they’re becoming more important. Clear, well‑structured content; journeys that reflect real user needs; collaboration with policy; plain English; transparency; consistency; these are still the foundations that make services trustworthy and safe.
What’s changing is that we now need to design with the expectation that much of what we publish will be read indirectly, atomised, summarised or reinterpreted by systems we don’t control. This means thinking differently about the resilience of the content we produce. If a paragraph on one of our pages is lifted out of context, does it still make sense on its own? Does it say something that’s safe and accurate even when separated from the journey it belongs to? Can it stand alone without misleading someone who might never see the full service? These are difficult questions, because government services are rarely simple enough for a single answer to be the whole story. Yet if AI systems are going to present fragments of our work in isolation, then we need to ensure those fragments are robust.
This also means working more closely with policy colleagues. Much of the nuance that could be lost in an AI‑generated summary is key policy: eligibility rules, safeguarding considerations, rights and entitlements that depend on specific conditions. If we want AI tools to represent this information responsibly, the underlying policy explanations we publish need to be concise, unambiguous and consistent across channels. Early collaboration between design and policy becomes not just useful, but essential.
One area we haven’t explored yet, though it’s increasingly on our minds is testing our services through AI systems themselves. Just as we test journeys with real users, we may soon need to test them with the AI models that interpret our content on behalf of users. This isn’t about validating the AI; it’s about understanding how faithfully our content is being surfaced, where it is being distorted, and what happens when someone relies on an AI answer instead of visiting a service. It’s early days, but it feels inevitable that this will become part of the design and assurance process for public services. We’re interested to see what patterns or guidance may come from GDS’ recent GOV.UK AI Studio work.
What makes this moment particularly important is that it touches the heart of what user‑centred design is for. Much of our work is built around supporting people who do not arrive with perfect knowledge of their needs, who do not know the terminology, who may not understand the system they are stepping into (because they shouldn't need to.) If AI‑mediated agents or answers become the dominant entry point, we need to be sure that people who lack confidence or familiarity are not disadvantaged further. We cannot assume that the most complete and supported experience will be the one users see.
The challenge ahead
This isn’t a challenge any single department can solve alone. The move towards AI‑mediated access is happening across the whole digital ecosystem, not just in pockets. The risks are shared, as are the opportunities. We need cross‑government conversation about what this means for safeguarding, clarity, accessibility, statutory accuracy, measurement, accountability and the way we structure information at source. We need to think about what patterns or conventions might help us create content that is “machine‑interpretable” without losing the human intent behind it. And we need to keep hold of the idea that design in government is not about answers alone; it’s about meaningful, safe and supportive journeys that help people understand their choices and needs.
The shift we are seeing is real, and it will shape the next decade of public services. But if we approach it thoughtfully grounded in our core principles, and open to new ways of understanding how users encounter our work then there is an opportunity here too. AI may change the pathways into government services, but with the right foundations, we can still ensure that what people receive is accurate, trustworthy and rooted in genuine user need. And if we share what we learn with one another, we can navigate this transition together.
If you’re seeing similar patterns in your own services, or asking the same kinds of questions, I’d be keen to hear from you in the comments below. This is a moment for collective learning, and the earlier we begin the conversation, the better prepared we’ll all be.
You can also follow updates from the GOV.UK AI Studio; and if you are a Civil Servant you join the #ai-for-designers channel on UK Government Digital Slack.
17 comments
Comment by anne-louise posted on
interesting read on a problem I have spent much time pondering!
Comment by Robin Hayden posted on
Might the dip in traffic demonstrate that there is a user need for information to be provided without the journeys GOV.UK often uses? Whilst making information available through a journey provides a more digestible way of providing complex information and context, it can appear as an obstacle if one is in a hurry and just want an answer. Perhaps if GOV.UK was improved to be better at providing bitesize answers without the journey, but with context to highlight there are other things users might need to consider, there wouldn't be so big a dip in traffic.
Comment by Mark Edwards posted on
Hi Robin, I think you’re absolutely right. The drop in traffic does point to a growing user expectation for fast, bitesize answers without the traditional GOV.UK journey.
The real opportunity now is ensuring those short answers still carry enough context to be safe, accurate and genuinely supportive.
For me, that’s the core challenge ahead: delivering minimal information with maximum context. And that ultimately depends on well‑structured, well‑maintained content so that whatever route users take AI or GOV.UK they land somewhere trustworthy and up to date.
Comment by James Halliday posted on
Great article, thank you.
Makes me think about some of the patterns in Richard Pope's Platformland book, which describe making policy (e.g. eligibility rules) and an appropriate data accessible to the public and their AI agents. So they can make better use of these in a myriad of ways that it wouldn't be possible to foresee or feasible or design journeys for.
Comment by Rachel N posted on
I've just been testing AI responses to some common questions we're getting in the charity I work for about people's eligibility for a new allowance. The guidance from central government to local authorities is post-graduate reading age, and understandably difficult for us to interpret in order to advise people on their eligibility, but also, presumably, difficult for local authorities to design their local offer around. The AI responses are drawing from our content (good for us), as well as other sources, but giving confusing answers because it's drawing snippets from different places. All credit to the current AI I'm using though which is at least giving me a disclaimer and asking me to contact my own charity!
Comment by Mark Edwards posted on
We're seeing exactly the same Rachel, be interested to see how you're tackling this, we're currently looking at how we can better construct our content for AI and potentially building out hidden FAQ's accessible for AI but not visible on the site to users etc.
Comment by Frank Kinsey posted on
As a designer I feel this is a vital aspect for us to get a hold on, we need to test our services through AI interfaces and understand how we can build information models that AI can understand and interpret correctly for users. More than ever, managing accurate language and context is vital.
Comment by Mark Edwards posted on
Testing services through AI interfaces feels like a really important next step for us as a design community. It’s starting to expose how much of our work has been optimised for human navigation rather than machine interpretation.
Your point on language and context really resonates, we’re increasingly designing not just for users, but for how systems interpret meaning on their behalf.
Comment by Felix Cohen posted on
We're seeing the same traffic patterns at the Chancery Lane Project and asking a lot of the same questions - with legal content the importance of the 'whole' document is really crucial, but also we know that many users are using AI to ask legal questions and build legal docs themselves, and if we can insert climate aligned legal thinking that's a huge win for us.
We've made a start on this internally but one thing that's been a big deal for us is thinking about how we test our own content - using deepeval with 'hard' prompts our legal team came up with to see how content and design changes improve legibility for AI: https://labs.chancerylaneproject.org/2026/04/16/our-content-testing-with-for-llms/
And out of that we've just launched a wordpress plugin to output legible and (crucially) metadata and taxonomy laden Markdown specificlaly to agent traffic: https://labs.chancerylaneproject.org/project/wordpress-markdown-for-agents/
Comment by Mark Edwards posted on
Thanks Felix 🙂 this is a brilliant example, really appreciate you sharing it.
You point about legal content and the importance of the “whole” document really lands, it highlights a tension I don’t think we’ve fully worked through yet. AI is great at extracting and recombining, but a lot of meaning (and risk) sits in how content is structured and interpreted in context.
Really interesting to hear how you’re approaching testing as well using harder prompts feels like exactly the kind of thing we need more of to properly understand how services behave in these situations. The work on outputting structured, metadata-rich content is superb it feels very aligned with where things are heading in terms of making services more legible to these systems.
Comment by Dominic M posted on
Agreed in the main. Interesting and challenging problem.
I would love to see an expansion of the topics in this article with specific examples of how AI renders content from GOV pages, versus how we intended them to be. And maybe some stats on accuracy of content, versus user comprehension, etc.
e.g. what percentage of the time is AI providing the right answer? What percentage of the time is AI fulfilling the user needs set out by the page, or missing certain elements?
Comment by Dominic M posted on
Agreed in the main, however, one thing I'm, not sure I agree on:
"...AI tools, by contrast, only answer what they’re asked."
The way I use ChatGPT – it has learned a bunch of context about me and my situations. It often proactively puts context together and offers me info that I may not have known to search for, or noticed that it may have been a factor. So I'm not sure we can say that they only answer what they're asked.
Comment by Mark Edwards posted on
I think you’re right, and it’s something I’ve been reflecting on since writing the post. The way these tools are evolving, they’re clearly doing more than just responding to a single question in isolation they’re starting to build context over time and proactively shape what they present.
Your point about that more “joined up” understanding is really interesting, especially when you think about how that compares to traditional service design, where we’ve often assumed the user needs to navigate and piece things together themselves.
It also raises a bigger question I’ve been thinking about we tend to assume that the original content we publish is the “best” version, but that might not always hold true. If an AI system has a better understanding of the user’s context, there’s a real possibility it can reinterpret and present information in a way that’s actually more useful for them.
Feels like there’s something here around how we understand and test for comprehension vs accuracy, and how much control we should (or can) expect to retain over how services are experienced.
Comment by Elle Flinn posted on
I feel this is even more important now that Google is going "AI-first" by default (https://techcrunch.com/2026/05/19/google-search-as-you-know-it-is-over/).
Many people presumably understand that Chat-GPT and other AI-first services are wrong - but old reliable Google?
I've been testing this with a particular bit of policy that I'm very close to, as a user researcher, and worryingly, Google's AI often cites outdated information or information from sources with a conflict of interest (e.g. LinkedIn posts from those with a product to sell). Even worse, it sometimes hallucinates entirely and stops citing altogether. Will users be able to tell the difference?
Really interested to see how UCD tackles this as a profession, feels like the most central challenge of the current age.
Comment by Mark Edwards posted on
Thanks Elle this is such a good (and slightly worrying!) example.
The point about trust is a big one. People have built up years of confidence in Google as a reliable starting point, so when the interface shifts but the trust stays the same, it creates a real risk if the outputs aren’t as grounded as users expect.
What you’re seeing with outdated or biased sources being pulled in really highlights the challenge the system isn’t just retrieving information anymore, it’s interpreting and recombining it, which introduces a whole new layer of risk around accuracy and provenance. And as you say, it’s not always obvious to users when that’s happening.
It does feel like this pushes UCD into a slightly different space. It’s not just about designing clear journeys and content anymore, but thinking about how we ensure information is understood, represented, and trusted when it’s mediated by other systems.
There’s also something interesting in how we test for this your approach of testing with content you deeply understand feels like exactly the right instinct, as it exposes where things start to drift.
I’d be really keen to see how this evolves in your work feels like we’re all circling the same core challenge here, and there’s a lot we could learn from sharing approaches.
Comment by Joe Baker posted on
Really interesting article, Mark. And what you're describing here about click rates seems to tally strongly with research published elsewhere (e.g. Ahrefs [Feb '26]: https://ahrefs.com/blog/ai-overviews-reduce-clicks-update/ | The Guardian [July '25]: https://www.theguardian.com/technology/2025/jul/24/ai-summaries-causing-devastating-drop-in-online-news-audiences-study-finds | etc.). I like the look of what the Chancery Lane Project folks are up to, and will be digging in myself.
It strikes me that this new context pushes even further the need for user-centred design, strong content design and structure, and super-strong metadata, taxonomy, and schema to make sure that the material being captured in LLMs and used in generative AI responses is of the highest quality … but then machine readability and human readability become twin considerations, eh.
Will definitely be interested to hear how you and the DfE team respond to this changing landscape. I wonder what your thoughts are on the CMA's new targeted rules to increase trust and transparency in AI summaries, and whether you'll be considering these at DfE? (https://www.reuters.com/legal/litigation/uk-regulator-enforces-new-competition-requirements-google-search-2026-06-03/)
Comment by Mark Edwards posted on
Thanks Joe , the idea of human readability and machine readability becoming twin considerations feels spot on. For a long time we’ve optimised for one, assuming the other didn’t matter as much but that’s clearly shifting quite quickly. The challenge now feels less about choosing between them, and more about how we design content and services that work well in both contexts at the same time.
Your point on metadata and structure is a big one as well it becoming foundational rather than a “nice to have”, especially if we want services to be represented accurately.
On the CMA side, it’s definitely something I’ve been keeping an eye on. It feels like an important step in recognising that these aren’t just neutral interfaces they’re actively shaping interpretation. How that plays out in practice is going to be really interesting, particularly in terms of where responsibility sits between the service provider and the platform.
Within DfE we’re still early in shaping guidance, but it’s increasingly clear this isn’t just a content or channel challenge it’s more of a system-level shift in how services are accessed and experienced.