---
title: "Transcript: How Electric Twin Builds AI That Knows Your Customers"
webinar: "How Electric Twin Builds AI That Knows Your Customers"
date: 2026-09-23T14:00:00.000Z
duration: "1 hour"
speakers: ["Charlie Cowan", "Tom Burke", "Diana Panizo"]
topics: ["Grounding AI models in real customer data instead of generic averages", "How Electric Twin validates accuracy against real survey responses", "The Times case study: 10x research output with the same team", "Where synthetic audiences work, and where they don't"]
recording: https://youtu.be/o912CwBE1SQ
canonical: https://kowalah.com/resources/wednesday-webinars/how-electric-twin-builds-ai-that-knows-your-customers/transcript
source: Kowalah Wednesday Webinars
---

# Transcript: How Electric Twin Builds AI That Knows Your Customers

**Recorded** 23 September 2026  
**Duration** 1 hour  
**Speakers** Charlie Cowan, Tom Burke, Diana Panizo

## About this session

Electric Twin builds synthetic audiences from your real customer data, so you can ask questions and get research-grade answers in seconds instead of weeks. 

## Topics covered

- Grounding AI models in real customer data instead of generic averages
- How Electric Twin validates accuracy against real survey responses
- The Times case study: 10x research output with the same team
- Where synthetic audiences work, and where they don't

## Recording

https://youtu.be/o912CwBE1SQ

## Transcript

**Charlie Cowan:** If this is the first Wednesday webinar that you've joined, we get together every Wednesday at the same time, and we spend an hour to talk about something that is top of mind in the world of enterprise AI. So that might be something that we're using internally here at Kowalah, it might be stuff that is going on with our clients and the projects we're doing, or other information that's going on in the wider ecosystem and new launches. And today, absolutely thrilled, we're going to have another one of our Spotlight Webinars, and I'm going to introduce them in a moment, but we're really excited to welcome Tom and Diana from Electric Twin, and we're going to be talking all about the world of AI in market research, and so more to come on that in a moment. Our Wednesday webinars take a similar format, so this is what we're going to follow today. We'll spend about 40 minutes talking about our main topic of the week, Electric Twin and market research. We'll then have our recurring segment of AI in the News, when we look back at the last 7 days and talk about any new model releases, new apps that have been released. And quite a week it has been, quite a 24 hours it has been, and look forward to covering some of the new releases from, Anthropic and OpenAI, and indeed, from SpaceX as well. And then we've got some time for Q&A. Now, what I would say is that if you're watching live, please don't worry about waiting until the end. If you've got a question about what we're covering, in the Electric Twin section or AI in the News. then feel free to ask your question. We're in a Zoom event, and so, depending on what device you're on. Maybe at the top of your screen or the bottom of your screen, you'll see there is a chat widget, and I'm bringing that up on my screen. And then there is also a Q&A widget as well, so let me open that up as well. And then, you can use either of those, and I'll keep an eye on them. And then also, Caitlin, who runs marketing here, is on, the call as well, and she'll keep, me aware if I miss, any questions. If you're watching on YouTube, as we know many people do, then just ask your question in the comments. And, we'll come straight back to you there as well. So, with that in mind, let's move on and meet the team from Electric Twin. So, very pleased to welcome Tom and Diana. So, Tom heads up Market Development at Electric Twin. Diana is one of the insight analysts there. And so maybe a brief introduction, A, to both of you individually. Tom, maybe I'll come to you first.

**Tom Burke:** Yeah, thanks, Charlie. Yeah, Tom Burke. So, I'm in the go-to-market function at Electric Twin. So, I effectively help organisations and customers onboard onto the platform, and then help get value once they're on the platform. And then obviously, like, share the story of Electric Twin across the market. My, I'm very… I'm quite fresh to Electric Twin, so almost 3 months in now, but my background, 10 years in tech. A majority of that time was in consumer insights, so I was helping organizations with, you know, customer segmentation, audience targeting, and research and insights work, and that's across the APAC market, some in the US, and also here in the UK.

**Charlie Cowan:** Fantastic. Thanks, Tom. Diana.

**Diana Panizo:** Yeah, Charlie, thank you so much, stoked to be here. I work very closely with Tom, I'm part of the Insights Analyst function, so I take care of anything to do with data or user onboarding into our main software. I also sit within the science team, so anything to do with validations, experiments, I contribute to as well. And I've been with the business for over 4 months now, and before that, I spent 3 years at a research lab doing a bunch of AI policy and behavioral science research for UN projects.

**Charlie Cowan:** Brilliant. Well, thanks very much, and I look forward to learning a bit more about what you guys are doing over the next, sort of 30 minutes or so. So let me just sort of set the scene. I'm coming to this as a sort of non-educated person around what you guys are doing, so I'm looking forward to learning more. The other day, I was on the train on the way into London, from, where I live near Bath. And about halfway along. a guy was coming along the carriage, and he said, oh, do you mind if I ask you some questions about your journey today? And I was like, oh, go on then, thinking this isn't going to take long. And so the guy sat down next to me, opened up his, I think he had, like, a ruggedized iPad or something. And then proceeded to bombard me with 35 minutes of questions about why I was going to London, and do I normally take this train, and how often do I take the train, and where do I normally get off, and when did I buy my ticket, and all of this, as everyone else is looking at me going, oh, you shouldn't have said yes. That's ruined your trip to London. But, so for me, this is my, sort of interaction with market research. I'm the customer, and the company, in that case would have been GWR, is trying to gather, data. And so maybe that sort of opens up the conversation of, you know, what is Electric Twin? What is this concept of a synthetic audience? And, how is that different from, this guy bombarding me with questions on my 6:40 train to London?

**Tom Burke:** Yeah, so I think, and a good place to start as well, I think in the market, a lot of… a lot of organizations are defining synthetic in different ways, and there's been a lot of… competitors coming in the space, and yeah, like, a lot of noise around this industry, because I think, you know, research is probably one of the areas that hadn't been disrupted as much as others we've seen in this space, so actually, like, a really exciting point, and where it is. In terms of what we do at Electric Twin, I think a good place we always like to start is actually in terms of, like, where we started and how, and it's actually part of the reason that I joined, because I think you know, when you're joining a company, you want to… you want to back the story and get involved in the mission. And, so there's a bit of background. So our two co-founders, two founders are Ben and Alex, they… they actually met during COVID. And they were both working at Number 10 Downing Street, so they're very different characters. Alex's, military background, so he's 20 years in the British Army, commanded task forces in combat operations, and he was asked to come into Number 10 as the Director of UK COVID testing. Ben, on the other hand, so he spent a decade in physics, so he worked at Faculty for a bit, and then went into polling, and he ended up at number 10 as the chief data advisor to the Prime Minister, and it was actually at that time that they were working through that. They were struggling to find, like, the right tools to understand human behavior and mitigate the pandemic, just to sort of, like, see how the public would react, right? to your point, Charlie, like, they probably didn't have time to get people out on the street and pulling them out from trains and running some research and running some surveys, so it was how can we make these fast decisions, at, you know, at scale? And if you just translate that issue to the commercial world, like, organizations are always constantly needing to make fast decisions with a few options. They can run that traditional research, they can use previously used benchmarks, so they've run research in the past and just resort to those insights. Or, very often that we see, they're just going by gut instinct. There's a lot of decisions that don't really warrant the budget that go into full-scale research, and then those are just made in the boardroom and going by that gut instinct. So that's where, effectively, you know, Electric Twin was born out of that initial problem, so what we do is we give customers the ability to, for anyone in the organization, to ask any question that they want to the audience that they care about, so often they're their own customers, and then get answers back in seconds. So. We, effectively, we, you know, create synthetic audiences, which, in really simple terms, is an AI model of your audience built from real data. So, it just means that it's an always-on audience, and you can get accurate answers back in seconds, so you can test ideas, and at a speed that just isn't possible with traditional market research.

**Charlie Cowan:** And so when we're thinking about the types of companies that might have an audience, obviously, public sector is one, you know, all of your citizens are an audience, but when you think of Commercial businesses. What kind of industries, are you, are you seeing?

**Tom Burke:** Yeah, so we typically work with large enterprise with, like, strong data and a constant need for, you know, some of these insights and decisions. I think often insights teams are… they're not really… they're not growing. If anything, sometimes they're shrinking, and they're also getting bombarded with a lot of questions from the wider business. In terms of industry, we've done very well across media and publishing, so people may have seen some great success stories we've done with The Times. We've done a lot of work with Virgin as well, so we're seeing a lot in that… in the travel space, especially around, like, that loyalty use case and loyalty programs has been a real one. Also, insurance and financial services, you can imagine often they have, you know, rich information on their customers, but often they don't interact with them as much as they would or they would like, so there's an ongoing need for those insights. So, we're working across a broad range of industries, but I'd say a lot of the time it's a large enterprise that have that ongoing need for constant feedback and understanding for customers that we normally find we do really well in.

**Charlie Cowan:** Great. What, if you want to share your screen, and I'll probably have some more questions, but just to talk about this concept of this audience, because I think this is the thing that, people will definitely have questions about, you know, How do you create that, and how is that reflective of a real audience?

**Tom Burke:** Yeah, for sure. I can just pass to Diana. Diana, do you want to just cover how we build it, and then I can share screen.

**Diana Panizo:** Yeah, definitely. I mean, I really enjoy the example that you gave, Charlie, because I've also been the person on the other side asking the questions, and I can tell you, it's not fun for us either. We know nobody wants to be answering these survey questions. So to talk through how we actually build the synthetic audiences, think about it as three different pieces of information that go into it. So the first one being your seed data. This is, structured data that you've put into the field, so it can be a recent survey that you've run, mixed with our LLM context, so we use, large language models to generate those personas, and then a third piece of that is that social science research that we've built our entire brand on. So we work very closely with institutions like the London School of Economics, with our scientific advisor, Michael Muthukrishna, to understand how we can better model these populations and how they differ across different countries, which is something that really differentiates us in the market. So those are the three main things that go into building an audience, and we're always really happy to work with our clients to understand what sort of data they have available, and if they don't have the specific data that we're looking for. How do we make it so that they do? We can make data accessible, we're very happy to invest in data sets to make our own portfolio more relevant as well.

**Charlie Cowan:** Okay, brilliant. So, take the example of the Times, because you mentioned that, and they would already have data about their subscribers. How long they've been with them, maybe their age or demographics or anything like that. So that is what seeds their synthetic audience, because they've already got that to bring to the table.

**Diana Panizo:** Exactly, and we have found that a lot of the organizations we work with already have this data, so it's the ability to create that always-on. Often you derive insights from field work that you run, but you don't do as much with it as you should. So this is really where synthetic comes into play, and you mentioned survey data. That's what we called our seed data. The reason why this is the main piece that goes into building these audiences is for validation. We find it that it's super useful to benchmark against the ground truth answers, and that's really how we are able to defend our validation, and we can talk towards the validation piece a little bit more as soon as we look into the platform.

**Charlie Cowan:** Yeah. Awesome. Yeah, Tom, I don't know if you want to share your screen, and…

**Tom Burke:** Yeah, for sure. I'll just… Hopefully I won't have issues on Zoom.

**Charlie Cowan:** There we go, we can see.

**Tom Burke:** So, again, and this just gives you a bit of context into how it's used, and I'll give a very… brief and simple demo. But as you can see, really simple interface, so you have a few options when you're trying to conduct some initial research. So, in the platform, your audiences will sit and your data will sit within the bottom left-hand corner. And just to expand on Diana's point, is that we often get the question of, like. do I have the right data that's relevant, or what data shall I use? One place that we always like to start is we start with, like, the use cases. So we're trying to understand, you know, what are you trying to solve for, what research you want to run, and what are the areas of the business that will be using it, and then we look to see. What data and what, you know, research is going to be relevant to build these audiences. a good way to think about how we support across, like, different use cases, I'd say, is that there's sort of three broader buckets. The first is like, early-stage ideation and, like, concept testing. So if you think about, we've got these ideas, and we want to validate which area we need to go and explore and go down, it's really great to validate those early-stage concepts. We can do creative testing, so both imagery, video coming onto the platform soon as well, and you can upload stimulus, so if you want to test different creatives against each other with the synthetics. And then the third is just broader audience understanding. So if you just want to get an overall view on, like, generic behaviors and motivations towards your specific offering, a specific product, it's really good just to get that pulse check across, yeah, across your audience set with synthetic. So I'll just show you how, yeah, how it works in terms of, like, real basic functionality. So… platform here, you've got a few areas that you'd want to run down. So we have a chat function, so often, if you don't have a really clear idea of the exact questions or research you want to run, you can use our chat function to guide you through some of that. So it could be like, look, I want to try to explore, you know, why people commute to London, or want to understand what the reasons for that, and how you get through to work, etc. It'll then guide you through that, so it can Identify what audience is gonna fit, depending on what data's available, and then it will guide you through how to, like, formulate a lot of those questions throughout the survey. We also have autonomous capability releasing soon, where we've heard a lot from teams that, you know, sometimes we just want the raw insights, and we want the key takeaways. So you can upload a brief. our agent will run all of the research in the background on behalf of yourself from that brief, so multiple survey questions, multiple focus groups, and then end up with a really rich report with some key takeaways. And the other part is you can then jump straight into a survey or straight into a focus group. So, I'll just run one now. This is live, so no judgement if any. But really simply, here's where you can select your audience. So, as Diana mentioned, when we take the seed data and build the population, they're available within the platform. And so, for this example, I'm just going to use Attention Economy, so this is just our, like, national representative, survey on broader attention of the UK population. You can add a stimulus, so this is where you can start testing some of that imagery, or you can upload some text. This is where I was talking about some of those… we were working with a large media organization where they wanted to test a few different concepts of podcasts. So they upload the concepts of podcasts and ask questions off the back of it. Or we can just go straight into asking surveys. So, if we run through some of the questions, I'm just going to go quite broad as we're using Attention Economy. I'm just going to say, where do you primarily listen to music? We can actually create sub-segments within the, audience as well, within the group. So we can split this by… again, these can be sub-segmented however customers like. We have organizations that have segmented their customers with their own internal segmentation. They have, like, high-value cohorts that they can then drill further into in the platform. this, I'll just use Millennials, Baby Boomers, Gen Z, and then where do you primarily listen to music? So, auto-generate responses. What will happen is, the AI will generate the best responses given your question. You can remove any if you don't like, or just keep it simple and say CD, radio, and streaming, and then we can run that survey. So effectively, all this is doing now is then that question is going to be run against the initial audience set. And then it will produce, yeah, produce a response. You can run multiple survey questions at once, you can upload a bigger survey. backing up Diana's point, as she mentioned earlier, is around, like, confidence. I think one thing… the common thing we get with anything, you know, I think anything AI-related, right, is, like, how do I trust this? Like, how do I know that, you know, I could be confident in this? So, after some feedback from a lot of customers, we've implemented a confidence score, which effectively gives you a confidence on a few areas that I won't dig into now, but we've got a load of, documentation on… it's effectively how well this covers the initial seed data, and then also, yeah, how confident we are that this is directionally accurate. So there's a lot of different, tiers of the confidence scores. So yeah, you get…

**Diana Panizo:** Jump into… to add a bit more on the validation. So again, that's the number one question we get. How do I know this is accurate? How do I know I can trust this? And basically, for every dataset that we onboard, we do what we call a holdout evaluation. So, we train our models on 90% of the data, and redact 10% of the questions. And effectively, when we create that synthetic audience, we then test ourselves. to see how well we answered compared to the ground truth. So we have that really, sort of tangible. comparison that you can get from the ground truth and the synthetic audiences, and we've run over 50,000 evaluations, in most of the countries in the world, and we do it for every single dataset that we onboard. So, we also have an internal judgment that if we don't think the seed data is good enough to provide a very highly accurate prediction model, we won't be onboarding it.

**Charlie Cowan:** And, you were mentioning, Diana, that before Electric Twin, you worked in, yeah, in that research world, and, I've been part of focus groups in the past, and a few of these things where. Yeah, there's a lot of science that goes into, well, how does the person that's asking the questions not steer what's coming out of people? How do you make sure that you don't have groupthink because of the person that's happy talking? And this is something that I think across, broad swathes of AI, is like, how do I trust, how do I trust? things weren't all roses the way it was before. So maybe you can talk a little bit about that, about some of the challenges of how this happened in the real world.

**Diana Panizo:** Yeah, you're exactly right, and I think there's two different ways you can think about it, and to this day, I don't necessarily know which one is the correct one, but you can have synthetic research where you completely block out all of that noise that you would see in the real world, like the genders within a focus group, how they influence how different females versus males are gonna feel within that dynamic of a group, how the order of people's responses influence, answers. now the question is whether you want to simulate that within synthetic, or you want to block that out. So, we do a lot of evaluations and science experiments, internally to understand what the best quality insight you're going to get from these focus groups and survey data. And I think it's really crucial to understand that this isn't completely replacing traditional methods. Like, it is really important to continue talking to your customers, and that human interaction is key. That's not what we're trying to say. What we're saying with synthetic audiences is you can iterate at a much faster rate, at a much lower cost, and uncover those questions a lot quicker, so that then you can also communicate that to your actual consumers. And I think that's really what we're trying to, get to with Merlin here.

**Charlie Cowan:** Brilliant. Well, I'll have some more questions in a minute about, sort of the trust side of things. A couple of things that you showed there, Tom. So one was about building the audience, so, it sounded like you've got some pre-built audiences, that… you know, even if someone wasn't going to load up their own data they could start from, are you able to talk to that a little bit? What kind of audiences have you already got, and whether that's you, Tom, or Diana?

**Diana Panizo:** Yeah, so we get two different types of clients. Those who have very rich, broad data of their customer base, or whatever the end user that they want to test is, and those who don't. And for that, we have two options. We either partner up and help you get that data through commissioning through a third-party provider, for example, or we use one of the off-the-shelf ones that we have internally. So the one Tom is using is a perfect example of maybe a telco company that wants to look at a broad audience and doesn't necessarily have anything brand-specific. We also have things within the water sector, broader FMCG attitudes. We recently did a really interesting one in Ireland about fast food consumption. So there is a really wide range of audiences, and the best thing about Merlin is that you can get as niche as you'd like. We even have one on business leaders and insight leaders. Actually, funnily enough, we commissioned a survey for ourselves to know how to improve our own products, so we asked synthetic audiences, what do you think about this new feature in Merlin? What do you think about, the layout and UI of focus groups. So we really are using our own product to improve ourselves.

**Charlie Cowan:** And just so that I'm clear, Merlin is the name of the model, or the product?

**Diana Panizo:** Exactly, it's the name of the product. We have another product that takes in unstructured data, so Merlin is the name that we use for this one.

**Charlie Cowan:** Okay, awesome. So, so there are different audiences that someone could choose, and then another thing that you showed, Tom, was about the stimulus, which I thought was interesting. So. Are you able to talk a little bit more about that? Like, what is the thing? And I'm, if I think back to… you know, seeing people standing outside Sainsbury's with their clipboard, this is where they might be, I'm assuming, showing a picture of a new pack of nappies or something, and going, right, here's a new design of nappies, you know, what do you think about that? Is that along the right lines?

**Tom Burke:** Yeah, exactly, and you can get… I think the interesting and fun thing about being in this world as well is, like, we're actually seeing new use cases for, you know, for organizations using it, and saying, oh, can we use it for this? And we've actually tested this within the platform that we hadn't, like, we hadn't seen before. But some of the key ones is exactly like you've mentioned, so it could be anything from, you know, creative, content. that is about to go out, and they want to just test, some of the imagery. It could be on, like, an out-of-home billboard, so they want to just test some of the key wording that might be on that billboard before they make that key decision. We've also had some interesting use cases on looking at, like, new mascots to support the business for a new summer ad campaign, and the design of those mascots. Like, does that resonate? Is that actually going to be linked to the brand, and what is… what are their customers' own affinity to that? to the different designs. So, yeah, we've seen a wide range of different use cases across that, but it's a really good starting point. Again, we can… I think where we're seeing it really valuable in synthetic in general is at each stage of, like, the research workflow. So even if you're really early stage, and you're thinking about two, you know, two to three very different creative concepts, and you want to understand directionally, it's great for that. And we're seeing organizations then actually conduct their actual real-world research in between, and if they do want to validate, they'll then come back to synthetics. We're seeing it In a really interesting way that is actually blending in with their normal research workflow, and synthetic is actually playing a key part along that journey, just to speed up all those, like, key decisions along the track.

**Charlie Cowan:** Brilliant. So, before we move on from this section, we talked about, audiences, so either sort of pre-built ones, or you're, you're loading up, based on what the customer's got, and, I've given a few examples that are basically B2C, you know, I'm on a train or I'm outside a supermarket, but, Have you got examples? Are you seeing around B2B? And so I'm thinking either, yeah, enterprise-type use cases, but also inside the company for employees. You know, if you're a 10,000-employee company, you've got an audience, that's your employees. Are you seeing people applying it in that direction?

**Tom Burke:** Yeah, absolutely. So we're working, you know, across the board. On the B2B side, yeah, we're working with some, you know, large tech organizations that, for that exact use case, so they can create those audiences off of their, like, key B2B personas. And then they're testing some of their new product concepts, or, you know, the roadmap items that might be due for release, and testing some of those within their key B2B audience, because you know, effectively, it's going to be the same. They're still, in terms of their buyers and their customers, they're still people that they can then run research on, so we've got some really yeah, really good success in that space. And it's definitely one that's growing at the moment in terms of that employee use case that we're still building across. We're having some conversations on some capabilities for there, because as you said. you know, large organizations, they, you know, conduct large-scale surveys across the internal pulse checks, and they need to make some key decisions across the organization. So, yeah, we're having more and more conversations in that space as well.

**Charlie Cowan:** Okay, awesome. So we've built the audience, we've given it a stimulus, and then you were just showing there about asking, well, a question, where do you primarily listen to music? Are you able to talk through a little bit about how you know, a client would be building… I'm guessing it's more than just one question, it's a series of questions, it's a survey. like, my half an hour on the train, is this, like, a one-off? Are people running these on a weekly or monthly basis? What's the interaction that someone's using to ask of that audience they've built?

**Tom Burke:** Yeah, yeah, for sure. So we… in terms of how we're seeing customers interact, very… really wide range. So some could actually just be jumping in, they want to get a quick response and ask their audience something that might be brought up in the boardroom, and they're jumping into Merlin and asking that, and they're running a single survey. While others are actually doing it as part of, like, a larger campaign or a larger initiative, where they're doing a lot of work in the platform over weeks to months. So, one example that I can actually… I can show you now through the platform. So, if you do have some of those initial questions, often you'll find if organizations, if they're running large-scale research. What comes back, you know, is often static in a survey, and you'd have always wished you'd ask some additional follow-up questions. There's always going to be some responses, and you go, that was interesting, I wish we had this follow-up question in that survey. And that's where I feel like there's a lot of value in a synthetic, because you can then iterate, and then change your direction as and when you need. So an example of that is what I'm showing you here. We've asked where you primarily listen to music. There's some examples that are no surprise. You know, Gen Z listen to streaming services, predominantly across the board, but if there's an interesting response in a particular cohort, like, for example, the Gen Z That, often primarily listen to music on radio. we can actually sub-segment that group, and then we can drill in further. So that's a key part of where organizations are using it, is to… is that iteration, and then drilling deeper and deeper to go, to go further into a… into a topic. So I can actually save that as a sub-segment of an audience based on their response, and then I can then jump into a focus group with the respondents that answered that question in that way. So, it's that jumping from the qual to quant to qual, and then you can really start iterating at speed between different groups and between different topics. So, I'll just show you from a focus group. We can then… Create a focus group. These are, like, the Gen Z radio listeners. We'll recruit the panel, and this will then pick 5 participants in this focus group that answered that question. We'll just remove a few of them. You can swap them around if you're not happy with the representation, and we can start the session, and we can just ask. Why do you choose… why… why do you, prefer. Radio. So this is where you can start asking those follow-up questions, and then dig deeper into a topic, and sort of asking them why, and the drivers behind that. So, the synthetic will… answer, they'll answer with each other, they'll agree with each other, they'll agree back to me. The key part, again, one of the values of synthetic is they don't get bored. you can ask hundreds, thousands of questions in a group, I'm sure, if you were in a focus group, Charlie, and you were there for 5 hours, and they kept drilling you over and over with different variations of a question, you probably won't be too happy. So the benefit of that is really, like, the experimental, process that you can go through, within Synthetic. So, look, honestly, it's free. Same for me, I'm not paying for something when radio does the job. And you can then… you can even respond to someone individually and ask those follow-up questions. And a good way to think about this is if there's some interested insight from what you've run on that focus group. we're then actually going to spark, okay, well, this is going to be something around, you know, if they're looking at those free offerings, is there anything around, like, the convenience aspect they're saying when they're driving? And then when these new topics arise, and these new insights arise through these focus groups, it's then… they can then jump into a new piece of research and explore that topic even further. So I think that's where the value is trying to find new insights, and then being able to jump at speed and going from An initial high-level question, right through to exploring a new topic in, you know, in a few minutes, as opposed to waiting for long feedback loops in traditional research.

**Charlie Cowan:** So I don't know if this is a… the right question, or, But, like, who is P1? So, like, who… because it's very compelling to read it, and then I'm like, but who is P1? Like, is that in some data somewhere? Is it the model? If I did this focus group again, would P1… answer the same way. I'd love to know, maybe Diana, what you think about who is P1? When can I meet them?

**Diana Panizo:** I think the main way to think about it is that when we create a synthetic audience, it's basically a one-to-one matching. So if you give us a thousand respondents in a traditional survey, you're gonna get a thousand synthetic personas. And what this effectively does is when it's recruiting individuals for the focus group, it's going to that ground truth data and picking real people. So for example, here we have Gen Zs who prefer radio. It's gonna go into the seed data. And think about it as. the ground truth gives you a personality type for each individual, and then all the models around it enhance it so that it's always on, and it can answer questions that weren't in the original dataset. So P1 is… sort of an amalgamation of responses from the ground truth data, with a lot of LLM context around it.

**Charlie Cowan:** I love, I love the… I mean, we work… we use a lot of agents internally, and, like, once you get… once you get on board with it, and you're like, right, I get it, then it's very… it's sort of addictive. You'll be like, right, I want to go and, yeah, just keep using it. So, we talked a little bit earlier about, I guess, like, sort of evidence and benchmarking against, sort of real world. And I think on your website, you talk about how accuracy has improved by around 20% since launch, and it's now about 96% on the industry-standard measure. How do you, help a client get to that point of, like, okay, this is… Essentially, one-for-one matching, if we were gonna go and send someone on the train or outside the supermarket.

**Diana Panizo:** Yeah, you're exactly right. We've had, I think it's about an over 20% improvement in NDAM, which is the accuracy metric that we use internally. I think the market benchmark is something called 1-MAE, but ours is a little bit stricter. And the reason for that sort of uplift is, number one, because of the improvements in the quality of the commercial LLM models. you know, with the new release of Astra and the latest models that were released, I think, last night, we're seeing a lot of improvement, and we're continuously testing these to see what model is most useful for synthetic personas. The second one would be just better persona creation techniques. I think we spent the first couple of years, within Electric Twin just focusing on how we were going to create that architecture to predict human behavior, and this has evolved a lot. Things like images, is something that we dedicate a lot of time to, because the way that humans ingest images is very different to how LLM models, do. And then the final thing is. a better, like, post-processing, of the LLM outputs, and how, with the feedback from the clients, we can make sure that that's something that you see in the platform. So if it's a client that comes to you and is like, this all sounds great, but how do I actually present this to a stakeholder? I can guarantee the week after, you're gonna have something in the product that's gonna help you do that, whether it's being able to cite your synthetic, synthetically generated results. having a branded deck, etc. So, I think those are the three main things that have led to that uplift, and as I mentioned, we do hold our evaluations with every single dataset that we onboard, and accuracy is something that we're very transparent and honest about, because, it's something that we really do pride ourselves on.

**Charlie Cowan:** Yeah, I mean, I think when we're building agents for clients, or, you know, wherever anyone's using AI, to take your number of, sort of, 96%, It infers that the human is 100%, and I don't think that's correct in all walks of life, whether it's development, writing blog posts, anything. So, the goal is not 100%, because the humans aren't there. And then you… and then you start to think about… what can you do now that you couldn't do with humans? And so, I think about the poor chap that was interviewing me on GWR. Well, the cost of that, the number of people that he can interview at once. The length of time that it takes to plan and to execute, that campaign over a period of months, and then as you're demoed, okay, you get all the results back. And he would go, right, here are the results, and someone says, but why did they get on at Bath? Oh, well, I can't now go and ask them all, unless we're going to do another survey. So, with, you know, with agents or any use of AI, so on the one hand, like. what are we measuring? We're not measuring ourselves against an infallible thing of humans. And secondly, we can now do loads of stuff that the humans couldn't do anyway. And so I'd be interested to know, like, I can feel that, let's say with my own use of agents, once I've got the confidence in what we built, then I'm all in, and I'm using that thing just relentlessly, because it's available, always on, and I can do stuff. So, are you finding that with your clients, that, yeah, there's a bit of this sort of tender, gingerly, oh, just checking, you know, am I comfortable, am I comfortable? And then one day, the switch flips when they go, right, I believe it, now, let's just ask.

**Diana Panizo:** Did you want to go ahead, Tom?

**Tom Burke:** I was just gonna say, it's exciting to see customers on that journey, because with any new technology, there's, like, skepticism, or there's a bit of nervousness around exploring, and I think often, quite early, I think we can always talk about the validation, and we can talk about our accuracy and our methodology. But a lot of the time, we say, look, just try it, and just test it out, and then you can then get to grips and start using. Often, we find that there's, as you mentioned, Charlie, there's an aha moment where I was actually speaking with a client a few weeks back, and they were talking… we were jumping in a focus group around a topic that they were actually currently talking about in the business, and one of the insights leaders shared, like, their hypothesis, and one of the responses from the focus group was. almost verbatim of what this insight leader said, and it's those little moments where it's like, okay. this seems quite accurate, and that trust then builds up, and I think there's a few things to that. It's one, it's then just, like, testing and trialing out the platform, and then two, I think it's also, like, the support wrapped in with the team to make sure that we provide, like, best practice on usage and how well it can be used, and how it might not be best used. But Diana, I don't know if you wanted to add anything to that.

**Diana Panizo:** Yeah, I mean, I think you covered pretty much everything, but just to go back to that human noise, you're exactly right in that if you give the same individual the same survey twice, they're not going to reply, exactly the same. So that's kind of the benchmark that we use as well. And I think it's also really important to understand What this enables, and it's the fact that you can bring so many voices forward that if you were that individual in the train trying to get people to answer your survey, you might not get to as many respondents as you would with synthetic audiences. So it's really important to voice all of those different demographics, genders, ages, and make sure that you make that insight available to the end consumer as well.

**Charlie Cowan:** Yeah, I just think, yeah, in my years of work, I'm probably counting on two hands the number of these types of exercises that I've seen. Obviously, I'm not running an insights team, but you know, it's a handful, because the expense, the time, are we going to do this, are we going to spend X tens of thousands on doing this? Whereas, now you've got that ability to just, well, let's just ask, well, let's just ask, once you've got that confidence. Let's look at it from a slightly other angle. Obviously, AI is going great guns. People are rolling out ChatGPT, they're rolling out Claude, they're asking it, absolutely everything. Fix my washing machine, fix my car. Why don't I just ask ChatGPT, is this new nappy design gonna fly?

**Diana Panizo:** I really like this question, and I think we get it more and more every time. And I think a misconception that people have is that our model is just a wrapper. As I mentioned, we spent the first good bit of developing Electric Twin focusing solely on the science. And we have tested our model against your commercial ChatGPTs, and we found that we did one very interesting experiment where we compared ChatGPT's answers to Electric Twin models' answers on the Gallup World Poll, and we found that directionally, ChatGPT does give you the correct answers, but Electric Twin's models, on average accuracy, beat every single question across all of the categories and countries. You're also going to find that your commercial large language models, first, they lack a lot of context. They're going to provide one generalized voice, whereas what we do is we model different behaviors, and backgrounds within your seed data. Another thing that you're gonna get is that ChatGPT's responses are overconfident, and they're nuanced, and they have a lot more variance than what you're gonna find with our prediction models. And I think the biggest piece to all of this is that you can't validate your commercial LLM's predictions, whereas you can validate ours, because they're built on something that is tangible and has happened in the real world. So I think those are the key points that I would make as to why our engines are a lot more powerful at creating predictions than your classic ChatGPT and Claude.

**Charlie Cowan:** Tom?

**Tom Burke:** Yeah, one thing I'd add is a common topic is, you know, the concept of, like, shadow AI, and… on that, we're finding that organizations are using, you know, these LLMs for these types of questions day-to-day, and I think recently we've had more and more conversations from leaders across insights, but also products, commercial, marketing, that, you know, they're using these tools and actually making these decisions. Anyway, from day to day, so they're actually wanting to look at, well, how can we actually use it based on our own data, with our own audience, with an accuracy that we're confident in and can actually make their decisions? So a lot of the conversation we have now is, like, a lot of this stuff is actually already happening. Whether the leaders in the business know it or not, I'm sure that people are punching in on Claude on their own personal computer, asking a quick question, so a big part of that is actually how Having confidence and having an approved tool in place that they can actually, yeah, use across the business that's, that's been approved, yeah.

**Charlie Cowan:** Yeah, yeah. We're… and a lot of our projects we're working on are to do with forecasting of some format, and it's a similar kind of thing. You know, if you ask Claude or ChatGPT, tell me what we are going to sell, tell me how many employees we will need in the… you'll get an answer. And then it's about how do you sort of evidence how you've, how you've got to that, and I think that's the… The gotcha, with this kind of thing as well, is this nappy design gonna fly? You know, well done, you know, you got this! But it's not actually, you know, built on any science behind it. And then I guess the final thing, just to wrap up this, section, is around… the need for real-world market research. Is the clipboard consigned to the bin, or, where do you see people, complementing synthetic audiences with real-world, research?

**Tom Burke:** Yeah, there's always going to be a need for real-world research, and you're always going to need to talk to your customers and talk to humans within the journey. There's obviously some of the obvious use cases around, you know, if you're, releasing a new recipe and you want to take, you know, show a new chocolate bar, you're not going to ask a synthetic or can't on the different… and taste in different recipes and different methods. But one thing that I think that, we've seen a lot, and I touched on some of it earlier. Is how we've seen it really, like, complement and speed up, like, the full journey. And are there areas in the beginning phases, in that early-stage ideation, where you can really speed up and validate what you actually spend money on in real-world research? And then also, obviously, post that, speeding up that decision process after you've conducted the research, and then figuring out to validate the last minute on those key decisions. Like, an example of what we've seen, we've seen one organization look and They're reviewing a lot of their loyalty programs and the partnerships. They receive, like, a big list of potential partnerships, so, you know, 40, 50 partnerships and brands that they could link up with. Now, that would take a lot of time to conduct real research to see what's going to resonate on that… what's going to resonate with their customers. So they used synthetic to effectively, like, shortlist those brands down to 3 or 4, and then they conducted real-world research on those three to four brands to then validate those, and then spend the money where it most mattered, whereas typically that would have taken, one, a long amount of time and a lot of effort, so we're really seeing it complement that full you know, research workflow journey at every stage, because as I said before, like, that speed of decision is becoming more and more important.

**Charlie Cowan:** Yeah, I'm thinking of, episodes of The Apprentice, when each of the teams goes and sits down with some primary school kids and, you know, gives them the toy that they've designed, and the kids are like, absolutely not using that. Something that's, like, physical, where you need to put it in someone's hands, and sort of for them to react to it, I can imagine how that complements, Yeah, the synthetic work. Brilliant. Thanks so much. I guess in sort of wrapping up this section, is there anything else that you would, you know, want to pass on to anyone? How do people get in touch with you guys? Anything else you wanna… wanna add?

**Tom Burke:** Yeah, look, the marketing team do very well in terms of the content we put out. We're always showing, what we've done in the market. We had an interesting case study, that we've… we've put out recently with the partnership with PitPat, so you can see Electric Twin on LinkedIn, ElectricTwin.com, so yeah, you can follow us there, and, yeah, we can… you can always book a demo through the, through our main website as well, and you'll get in touch with the, with the team here.

**Charlie Cowan:** Yeah, awesome. Well, thanks, Tom, thanks, Diana. I'll come back to you in a minute, and we'll just, go on to our recurring AI in the News segment, and you may have some comment on this. I always say, you know, each week, we'll do our AI in the News, and who knows what we're gonna have. In the next 7 days. And every 7 days, it just, it just delivers. There's so much going on. So I have to really, sort of, refine what I'm talking about. So I'm not going to talk about Muse, although I'd like to talk about Muse, quite a lot. The… the three things I'm going to touch on, two main ones, and then one extra, is new model releases. And so this happened last night, UK time, morning, on the West Coast. So, first mover was Anthropic, who launched Opus 5.5. And so the short, update here is that you can get Fable-level work, so Fable is their top-level model, at up to 40% less cost with their new Opus 5.5, model. But more than that, Opus 5.5 is also about 20% less than Opus 5 was, and so for anyone that was just using that, you're going to see this reduction. Now, this chart on the right is one that I like. It's kind of adopted by quite a few of the vendors. Because it shows two things that are important. So, the y-axis, is showing you, the success against certain tasks, at various levels of thinking. So, the orange line is the new Opus 5.5, and you can see from low thinking, up to max thinking. But then the important one is this logarithmic scale along the bottom, which is the cost per task, cost per attempt. And so you can see that going from $20 per attempt down to… down the… well, it's logarithmic, but less than one on the left-hand side. So the upshot here is you want to be up and to the left on this chart to be good. And as each one of these new models comes out, we go more up and more to the left. And so what you can see here is that if we compare to Fable 5.1, which is the green, we're significantly up and to the left. And against, what have we got? GPT-6 Astra, which would be in the top one. We're up and to the left, so this is all good. Now, I haven't put a slide on for what I think is the best highlight of 5.5, but it is better writing, better communication. Now, for anyone that was using Opus 5, up until yesterday, you will have had loads of, here's what leaves the room. Here's the next beat, and here's something that is worth naming. All of this gobbledygook that no one writes like that was coming out of Opus, and it was becoming, you know, a real problem for people that were trying to use, Opus for knowledge work. They've really, focused in on that, and certainly my, first, uses of 5.5, it is just way better at communicating more clearly, in less words, and so definitely one thing to keep an eye on there. So that was Opus 5.5, which came out late afternoon UK time. About 90 minutes afterwards… OpenAI unleashed their new models as well, and so I was, reminiscing of watching Braveheart many years ago, for anyone that's watched Braveheart, hold, hold, hold, they're sort of waiting, and then the moment Anthropic released something, bang, there you go, have a few more models. So, what OpenAI have launched is their… the rest of their GPT-6 series. So previously, they had had GPT-6 Astra, which was the top level, and then they've released Sol and Luna, which are the next one down, and the next one down after that. And so, they've dropped the price: Sol and Luna come in at half the price of the GPT-5.6 versions they replace. So, using the same, type of chart that we saw previously, we know that up and to the left is better. And so here you can see GPT-6 Luna all the way over to the left, and then GPT-6 Sol, which is the yellowish one, is further up, but more expensive. And of course, this doesn't have the new, Opus 5.5 on it, because it only came out about 90 minutes before. But generally, as I say to our clients. Every week, the models are getting better, and the models are getting cheaper. So, this gives you two things. So, one, the tasks that you think, AI can't do this, AI can't do this, well, the models are getting better and better, so they're bringing more and more complex tasks into possibility. But secondly, because the models are getting better, and the models are getting cheaper, it means that the models that, were cheaper before are now able to do more complex things. So, you don't need Astra, you don't need Fable, you don't even need Opus 5.5 to be doing what would have been perceived as more complex tasks in the past, because now you can be using Sonnet, you can be using Luna for that. And so… More and more, the lower value tasks are coming into scope. Diana, Tom, maybe I'll just ask you a question. Obviously, you are building, on top of, maybe these models, maybe other open source models. How do you see that as the models get better, it affects what you're able to do with your clients?

**Diana Panizo:** Yeah, I mean, it's a very exciting time in this race for releasing models. I think at Electric Twin, it's really key to make sure we're up-to-date with all of these new developments. I haven't had a chance to try the ones that were released last night yet, but I'm pretty sure that our science and engineering teams are very busy now looking to see how these compare with the past models that we've been using. And I think specifically for synthetic. We do see uplifts in the level of accuracy that you get, but it doesn't always mean that the most expensive or newest model is going to be the best one, and I think that's a really clear distinction that should be made as well, because in the same way that not always more data leads to richer personas, it's the same case for models as well. So, it will be exciting to take these on and see how they compare.

**Charlie Cowan:** Yeah, absolutely. I mean, my message over the last year has been that for the majority of knowledge work, the models are already way more powerful than you need. You know, for Lynne in finance to plan her one-to-one with her manager, she does not need Astra. So, the models are not the limiting factor for the vast majority of use cases that you might have. So, you know, no one should be waiting for newer models to come out. And secondly, the compounding effect that you get by just starting now and learning, as the models get better, you know, everything will get better. Oh, I… I said… let me just, come out of that, and I said I was gonna do two updates, plus one more. The other one is… I mean, this seems like a long time ago now, September the 21st, two whole days ago, this is so out of date. this was Grok 4.7, being launched. So, Grok from SpaceX, from Elon Musk, only a few months ago, you would not have really counted, Grok as being a, I think maybe, fair to say, a respectable enterprise tool that you'd be using, but it is now gathering huge pace with Grok Bot as being an agent that enterprises are using, and Grok 4.7, if I go and find… obviously, these are out of date from what happened last night. But in terms of, the benchmarks, 4.7 is right there, just behind the Frontier models, at a very competitive, price. So, definitely one for people to be taking a look at. So we covered a lot, let me just go back to present here and say, once more to Thank, Diana and Tom very much for joining us. Next week's webinar, we are going to be talking about WebMCP, which is a new proposed standard from Microsoft and Google, and it allows you to put tools onto your website, which means that when someone's agent comes a-browsing of your website, instead of it just taking screenshots and working out what to do next. you're actually able to guide that agent through explicit tools. So I'm going to talk you through that, and I'm going to show you a demo on our website using our AI cost calculator. So thanks very much, Diana. Tom, thanks everyone for joining live, and for those that are joining on YouTube, and we will catch up with you on our next Wednesday webinar.

**Tom Burke:** Thanks, Charlie.

---

This transcript is published by Kowalah, a UK Anthropic Implementation Specialist.
Webinar page: https://kowalah.com/resources/wednesday-webinars/how-electric-twin-builds-ai-that-knows-your-customers
All transcripts: https://kowalah.com/resources/wednesday-webinars/transcripts