Nik (00:05)
Welcome to AI and Design, where we explore how artificial intelligence is reshaping the world of design. I'm Nik Martelaro.

Dan (00:10)
And I'm Dan Saffer, and we're faculty at Carnegie Mellon's Human-Computer Interaction Institute. Each week, we break down the latest AI developments, dive deep into topics that matter to designers, and talk with fascinating guests who are right at the intersection of these fields. Whether you're a designer working with AI or an AI practitioner interested in design, we're glad you're here.

Nik (00:36)
On today's episode, we'll be discussing OpenAI announcing Frontier, a system for building AI coworkers.

Dan (00:42)
And in the age of AI, does the intuitive designer prove themselves to be more relevant than ever?

Nik (00:48)
And we'll wrap up by continuing our discussion of the value of user research with Dan's interview with our first special guest, Shane Johnson, Principal Researcher for Special Projects at Figma.

Dan (01:00)
But first, let's get into this week's top AI and design stories, starting with OpenAI's new Frontier. If you can build AI coworkers, what does this mean for design teams? Nick, can you tell us what Frontier is and does?

Nik (01:19)
Sure. So this is an article that came out on February 5th of OpenAI announcing Frontier, which is this system that they're giving as a way to help enterprises build, deploy, and manage AI agents that can do real work. And I think the goal here is that they're trying to build systems that create agents or specialized AI workers.

that can do certain tasks and they can do those tasks really well and those can be then embedded into the work processes at different companies. So one of the things to note here, this is definitely an enterprise product. They specifically talk about the different companies that they have been working with and that they're adopting these things. This is not as much right now at least of a consumer product, but I think it's really interesting

because it really shows that they're trying to get AI much more embedded in the different sort of work processes that are going on. And so from a company perspective, this is a way in which you and OpenAI can collaboratively build these agents to do certain tasks for you. Some of the example tasks that they actually give in their demo are things like customer service agents, equipment maintenance agents.

marketing campaign agents, root cause analysis. And so what they're doing is they're creating, these specialized AI assistants or AI agents that can take actions, they can take in information, they can take information in from the company, and they can take information in from say interactions that different people, whether it's customers or other workers within the company are doing to actually do real work.

Dan (02:55)
What's the problem that they're trying to solve here?

Nik (02:59)
think the main thing is, that a lot of companies, you have all this knowledge, you have your own systems, you have your own information, and that's often within your own systems, databases, Google Docs, Word files, intranet, what have you. And so I think the idea here is that OpenAI is working with companies and Frontier is this sort of way in which you can have that business context that you already have.

accessible by AI agents and then usable. Then on top of that, you basically build all of your different sort of use case agents. So these are the different things that are doing work. One of the things here is that I think they're trying to make it such that also you can expose different tools. So if you remember last week, we talked about MCP. And so I think

One of the things within internal company systems and within your own processes is you actually have to then expose those tools. Like, if there is a customer support agent system, you actually have to expose that to the AI agent. And so they think they're working with companies to make sure that those things are exposed, make sure that the AI agents that are within a frontier system can actually access those. The other thing that they're doing is that I actually thought was really interesting was they are setting up

good systems to evaluate these agents. Because the reality is that for as much as the benchmarks of all the AI agents that are out there, like Chachi, BT, Claude, they're getting better and better at these sort of general benchmarks. But real work actually has its own benchmarks. having an agent that's good at a bunch of things may not actually tell you how good it is at being a customer service representative for your company.

And so one of the other things that they're doing is they're setting up systems so that you can build your own evals, your own evaluations for say how good of a customer service agent is this OpenAI agent for our company based on our own data and based on sort of real world interactions. And from there, you can then sort of build up and optimize your system so that your agent works for you. So it isn't a general purpose. You're not just throwing general purpose GPT at this. You've actually

gotten the system to work in such a way that it's meeting your standards and your criteria.

Dan (05:14)
I don't care if.

AI can do PhD level physics, I care if it could answer 10 more calls during the day in my customer service. That's kind of what it's really aiming towards.

Nik (05:25)
Yeah, exactly. It's aiming towards those like business specific, like what is the goal for your company? What are you trying to get this thing to do? What are, how should you evaluate it? How should you actually judge if it's doing its job correctly? In the same way that you actually might have evaluations for your human workers. So in a similar way, right, you're checking to see like, is this actually, you know, working well? Are they able to resolve the problem? Are they doing it in a way where customers are happy?

or if you're doing other types of things. It's interesting. This is the kind of thing where within the code space, we've been talking about agents that do different aspects of programming and different actual agent systems. We've, of course, talked about skills before. And so this is sort of a mix of all these things in a way. There's these skills that you're giving these. You're using tools. You're exposing the tools that they can use and making sure that they use those tools correctly.

So that they're actually working with the right data and the right information, not just hallucinating and making things up. The thing that's, really interesting about this is the fact that they're trying to embed into sort of real world work environments with real world tools. It echoes what Claude had put out a couple of weeks ago with Cowork. The idea that Claude can now operate on your files, it can actually operate on

different systems on your computer. Basically, you could kind of try to use your computer to do stuff, which automates more of your work. And I think that building agents for that is kind of in this nascent phase, like of actually having the agents autonomously do this work. So what does that mean for design? Right now, they don't actually show any design-oriented use cases here. But I think that there are elements of this that

Dan (07:00)
you

Nik (07:05)
designers should start thinking about, right? What aspects of your job, could you potentially automate? Maybe that's a place for this. Also, how do you get your systems ready so that when you have agents like OpenAI's Frontier, like Claude Cowork, that can start working on files, working on other elements of your business systems, how do you make sure that those systems are ready to go?

Dan (07:27)
I was thinking as you were saying that just this does feel like a competitive play versus Anthropic, which seems to be very much going after that targeted work enterprise level types of things. And up until now has been very much focused on developers, but this kind of opens it up to many, many different kinds of roles, one of which is design.

Nik (07:56)
Right. And I think that this is an opportunity maybe for designers who are out there that are thinking about, could we utilize these tools and parts of our process, especially if you're an enterprise designer, like where could we use this? One of the things that's really interesting about this, and this is at the end of the article, is how they talk about the way in which they're deploying this is they work with partners and OpenAI is deploying

forward deployed engineers or FDEs. And I had to look this term up. So a forward deployed engineer is basically an engineer that comes out and works with you, like deployed maybe on site and then understands what you're trying to do and then helps you set up say this frontier system. The interesting thing about this is that the role kind of has a mix of user research,

need finding, spec writing, and then feeding that information back to, specifically in this case, OpenAI's research team to potentially make the models better. So they're sort of acting as this conduit, but it's a really interesting service-oriented role that also has some elements of design. Like to be a forward deployed engineer, oftentimes it sounds like folks have some programming abilities, but in this case, I mean, it could simply mean you have agent development abilities.

You maybe have vibe coding abilities. I mean, I should say we don't know exactly what the forward deployed engineers at OpenAI and what their training is. I thought this was a very cool thing to see from the perspective of, this is a role in which probably people who have design-oriented minds, they have a ability to understand and observe users. They have an ability to translate that into needs. And then they have an ability to basically work

with say the open AI systems and configure and develop these, this could be a really cool role for that. And it kind of reminded me of some of our conversations on sort of the need of user research. mean, this is clearly right. They're not using AI to configure these things. They're sending out people to work directly on site with these companies to get stuff working.

Dan (09:51)
what's interesting now is you could have designers or user researchers or product managers morphing into this kind of role because the barrier that was

the super technical part seems to be dissolving a little bit. So I do think it is a really interesting thing to think about from a career path

designers could start to move into these more, what have been traditionally technical roles.

Nik (10:20)
I'm actually wondering if this is even more of a exciting place for designers to play or where maybe this story has more relevance to designers at the present, maybe less on the direct, like, okay, what's frontier? How are you going to automate aspects of your work? But more of what is this role that designers can start training for? Because I agree, If we, as designers,

don't need to know as much about, say, writing code or have all of the skill and craft of handwriting code, but we're adept at working with these systems. We're adept at translating into product requirement documents and then utilizing code agents to build the systems if we are good at writing prompt systems. I mean, that's one of the cool things is a lot of this stuff is likely prompt systems and not even code systems. It's simply just writing good prompt systems.

⁓ for calling GPT or some slightly tuned form, an internal form of GPT. And so from that perspective, yeah, the barrier to entry for designing a system and designing workflows and being that type of designer, think has really been reduced. And so I think that's the thing that's potentially really exciting about this.

Dan (11:29)
All right. Well, that is OpenAI's frontier. Next, we are going to talk about an article that came out a couple days ago called The Return of the Intuitive Designer in the Age of AI. And this is an article by James Harrison. the basic premise of the article is that there are

designers who are really good at following their instincts that are very good at these kind of intuitive designs that they're making designs that are unusual, unique, but are still solving a problem. And

they can do it without a lot of sweat. And the way that they're able to do it is that they just have intuition. Talking about the creativity stuff that we talked about last week too. They're probably in that like top 10 % of ⁓ very creative people. And the idea behind the article is are these people becoming more and more important

in the age of AI because as we've talked about before when we were talking about impeccable is so much about AI is generating these things that are baseline They're not generic, but there's definitely a house style to them because they are trying to get the most likely answer. And the idea behind

James Harrison's article is perhaps these intuitive designers will flourish because they're not coming up with these very baseline expected answers. They're coming up with things that are unexpected and intuitive and leaps ahead of what's already out there.

Nik (13:14)
Yeah, I thought that this article was quite interesting. Also for its look back in history, one of the things that it talks about is how certain industries will basically go through a transition from being a craft-based industry towards one that gets mechanized and you can then start to automate aspects of things. You then begin to de-skill the role that's required. But then from there, right, certain aspects become de-skilled.

So the example given is automotive manufacturing, right? Making cars before the assembly line basically was still mostly a very craft work type job. Then once the assembly line came on, you could start to de-skill. You didn't need as much skill. And then of course, further and further in this point, a lot of automotive production is automated from a fully mechanized perspective, though there are so a lot of elements of human labor involved. In a similar way, design

is going through that. That's really what Harrison argues in this article. We're seeing that phase change in design. We're going through where a lot of elements, at least of digital design, are being mechanized and they're being automated. It is easy now to create generic, reasonable-looking front-end user experiences, reasonable software with AI systems, and it's only going to get better.

And so then yeah, then this question is, okay, well, What then do designers do? And then this is where, Harrison really talks about and kind of brings up this aspect of the translation from understanding what is it that people need, understanding what is actually good and not good, and making those judgments as being the important aspect.

of design. Actually, he brings up a quite provocative talk by Jenny Wen, who is the lead designer at Anthropic. in that talk, Wen says, don't follow this process that you've been given, which is interesting, because in a way, that process is one of the first steps towards the

Dan (15:07)
Mm-hmm.

Nik (15:11)
automation and mechanization, before we even had the design thinking process, we just had really good designers who arguably had an intuition for doing something. Then eventually we moved towards saying, well, no, no, no, wait a minute, there are methods to this madness. And we actually can make this a repeatable process, which is what businesses want. If everything is left up to pure intuition, it's hard to run a business that way. If your goal is to run and create products, you kind of do want to mechanize things. And we sort of did that.

with design thinking, design methods.

But at the end of the day, I mean, there is still this element. I think great designers, great design companies, they're still leaning a lot on

intuitively fits and what seems to work. Now one of the things of course to remember is that a lot of that intuition is built with years and years of practice.

Dan (16:00)
so Paul Rand famously when Steve Jobs came to him for the next logo did it very quickly and said here's your logo and Steve Jobs was like well this didn't take you very long and

Paul Rand was like, well, it took me a little bit of time, but it also took me 40 years to be able to do this.

Nik (16:18)
Dan, I actually wonder though, could this potentially supercharge people's ability to develop intuition because they can create so much more? Now they're creating in a different way, right? They're not maybe designing things potentially even by hand. You can of course still do this and then throw it into something and then actually get a works like prototype. But I mean, actually working with these systems and building more, designing more things, and then making sure that you're very

aware of and exercising your reflective loop. Why is this design working for me? Why is this not working for me? Going out and getting feedback from people, seeing if the design works out in the real world, trying again, failing again. But the fact that you potentially could create it even more. I arguably, I mean, in my class, for example, students do four projects and I tell them, they have four weeks, then three weeks.

to do their projects. And I think some of the students, when they get started, before they utilize these AI tools, go three weeks, three weeks to do a new product development, like a full thing. I mean, that sounds, how? By the end of the class, students are telling me, I designed three things because I realized I could go in all these directions. I didn't know what was really gonna work. And really, by the time I did the last one, it only took me a couple hours because

By that point, I'd honed my skills, I'd honed my curatorial eye, I understood what I wanted, and then I was able to basically work with the system to build really quickly. So, yeah, I actually wonder if this could supercharge people's intuition if they're very thoughtful about their reflective practice.

Dan (17:52)
Mm-hmm.

Right, think that's the key, right? Is to not take the first outputs and be like, it's done. And instead, to really push it and think about, how could I improve this? How could I tweak this? How could I make this better? I read an article a couple of weeks ago that was all about how a lot of design teams were using these kind of generative tools.

One of the things they said is that they do exactly what you just described. They generate a bunch of stuff and then they all sit down in a room and they talk about it. They debate it. They see what's good. They see what's bad. They reflect on it. And then they go and they do another round and then they do another round until they get what's right. They, they, they keep refining it. But the

thinking part, the reflection part, both right up front before they actually start doing anything, and the reflective part once something has been made, are both the really important things, not necessarily the production of it. Now that feels weird as someone who was taught for so long that so much of design is about the making part that there's

Design thinking, there's design making, then there's design reflection on what you've made. And so that is a little brain breaking, but I think that is a way that we're going to see a lot of work start to happen in the next, well, it's starting to happen now. It's happening now, but I think this may be the start of a new.

design process that we have. That's really not all that different from the old design process. It's just a lot faster.

Nik (19:52)
Yeah, I mean, there's a good story in Ken Kasianda's creative selection book. So this is about design at Apple. the story, the one I like pulling up often is the design of the iPhone keyboard, because actually they had all these different ideas of keyboard designs. It wasn't just a QWERTY keyboard, even though it seems like obvious, but they tried all these things. They tried like T9 style. They tried these like...

rotary dial things, they all this stuff. And then what they did is they tested it and they tried it within their teams and they realized this isn't working, this doesn't work, this doesn't work. And every time of course they made something, right, they learned something.

I'm excited by the fact that AI systems now allow you to create functional prototypes that effectively reach a level of fidelity that normally would take you at least days if you were adept, if not weeks. And you can now do these and crank them out in hours. Everyone on your team could do three of these and you can come to a meeting with nine or 10 different things.

The other thing too here is that no one is telling you like, you have to do this. I have students all the time go, yeah, I made five ideas, but I didn't like any of them, but I realized this is what I wanted. And then I just did that by hand, but I knew what I wanted by that point. And so the thing here is this ability to basically then again, it's honing, I think that intuition of what it is that you want. And then,

Dan (21:08)
Mm-hmm.

Nik (21:17)
you can recognize that and then start taking direct action much more quickly. And then you're working in a very purposeful way. Whereas I think sometimes also when we sketch or when we work, it is with an open exploratory way. This is just another way of doing that. Just a excedingly fast way.

Dan (21:34)
I love the idea of working to figure out the things that you don't want. We have students all the time making these, janky prototypes where they just crank them out based on ideas or concepts. And then they can start to see like, Ooh, this doesn't work or ooh,

I like this piece here, but I don't like that piece there. And then, yeah, you could then go and build those by hand if you really wanted to, taking all the pieces that you really liked. Or you can start to use the AI tools and start to say, like, give me that piece, give me that piece, and start to stitch them together. It's all kind of how you use the tool and

where the tool breaks. mean, we're still getting to the point where things like design systems are still, we're still learning how to incorporate them into different AI systems. And so having those like pixel perfect, objects that we're going to need for production is just starting to get there. And until then,

you may have to always be firing up Figma and playing with it, or you may want to, depending your proclivity And frankly, some things will be easier to do in Figma with direct manipulation rather than trying to describe it to a prompt. So I think the design process is a little off for grabs.

Nik (23:17)
You know, in talking about the design process being up for grabs, Jenny Wen, that is a lot of her kind of argument in her talk. The talk, think actually, if I remember it correctly, don't trust the process. You know, she really talks about this idea of sort of breaking the process. Of course, there's all kinds of stuff, in the talk, she does talk a lot about other things like, well, in my process, I still do this. I'm looking at user research data. I'm looking every day at...

what customers are doing, I'm learning from that and then coming up with ideas. you know, so one of the things too is I think sometimes people get a little too wed to the process rather than understanding that many of the methods we've developed as designers are components of a process, a process that you put together within your team and even within yourself. And so this is something now I think actually

these AI tools are maybe starting to push people to challenge maybe their assumptions of the process that they've put together to say, might I work in a different way? In the end, you're still going to have a design, if you're a designer doing something, you're gonna have some process. Just what order you do things in, what components you use, what tools you use, those might change. And I think really then the skill there, especially on this intuitive designer aspects of things is which designers recognize

how to construct a good process that works for them and works for their team.

Dan (24:42)
artists have had this notion of my process for, I don't know, forever. And I think maybe that is something that designers have to start embracing, that they are going to have their own process that may be very personalized and very individual to them based on very personalized tools that they can make for themselves.

Nik (25:10)
now you're making me think here, in like an interview, we show our portfolio, we show the work we've done, we typically would show, our process, especially when we talk to people, what user research did you do? How did you start thinking about, what were your intermediate artifacts, your user flows, these types of things? Are you now going to have to show and here's my tools, like, here's how I work, because I've now created

these custom AI systems, my own AI agents, do you bring that in? Because often we've relied on other tools and our skills and our abilities, right? You definitely, when you're getting hired today, they're gonna ask you like, hey, are you competent with the software that we use in-house like Figma? And they'd like to see that you know how to use it because you kinda wanna be productive. Now of course, any designer should be able to learn new sort of

Dan (25:50)
Right.

Nik (25:58)
generic design tooling. if you came from sketch, you should be able to move over to Thigma eventually. That's, you haven't lost your skills as a designer, but I do wonder now, are people going to be as, hey, show me your tooling, right? If you're augmenting yourself, I'd like to see what you do. And I'd like to see how you work with that to see if that's actually going to bring more value into our team. I have no idea. I'm sure people are asking.

know actually people are asking today, like, how do you use AI? But I actually wonder if there's going to be some kind of evaluation, not just of the end product, that you produce. Hopefully, because at this point, I hope that everyone is creating great design, but that you have this process and this set of tooling for yourself that is going to fit well with the company or that is going to make you really productive.

Dan (26:42)
yeah, here's all my custom tools that I use to be intuitive, to be creative. Yeah, it's really interesting.

We use the word in we use the word interesting way too much in this podcast, by the way.

Nik (26:55)
Probably we do, but...

Dan (26:56)
I

have to cut it out a lot.

Nik (27:00)
Hilarious, that's funny.

Well, so I think this is really interesting.

Dan (27:06)
There you

go, right there.

Nik (27:09)
What was going to say is, on that note, Dan, of custom tooling and how people are changing, I'd love to hear your interview with Shane and see what you learned talking about user research from out in the field.

Dan (27:22)
Yeah, let's end our conversation there. And here's my interview with Shane Johnson of Figma.

Dan (27:30)
so last week, Nick and I talked about user research and there was a bunch of questions and we were like, maybe we should have an actual user researcher here to talk to us about these things. So that's why we have here Shane Johnson, who is user researcher extraordinaire at Figma. Welcome Shane.

Shane (27:52)
Thanks, Dan. Good to see you again.

Dan (27:54)
Anything

I know it's been a while since we worked together at ye old tweet factory. since then you worked at Slack. You've worked at a bunch of places, right? Slack and now Figma.

Shane (28:06)
Yeah.

Yeah. I spent a little time in a startup. I got that experience as a researcher at a 30 person company, which is very interesting. Yeah. but yeah, most recently, Slack startup and Figma.

Dan (28:15)
⁓ that's fun.

How long have you been at Figma?

Shane (28:23)
This is gonna be my fourth year at Figma, which feels like forever and time.

Dan (28:25)
wow.

That's really riding the rocket ship. mean, when I went to config last year, was like just incredible to me, like the size and scale of it. Like every design conference I've ever been to, rolled into one, just the number of people is wild.

Shane (28:47)
I know. I don't even think of it really as like a conference for Figma anymore. Cause every time I go, end up running into people like you that I've met with or I know from my past, right? And like, it feels more like a homecoming for the design industry.

Dan (29:03)
Mm hmm.

One of the places where user research comes in is knowing, are we on the right track here? Is this really the thing that we should be spending our time doing? And that's why I've been so dismayed about

the entire industry seemingly cutting user research to the bone at a time where I feel like we really desperately need it. We have a new technology that we're all fumbling around with and we don't have enough user researchers to help us figure out how to do it right and what people really want and what people really need and how they understand working with the technology.

That's my mini rant there.

Shane (29:47)
Well, it's good to know that some folks still see a use for us. ⁓ I mean, yeah, it definitely has been an interesting five years. And you're right that I think research is often the thing that people look to cut because the impact isn't as obvious, right? A designer makes the screen that an executive ends up looking at.

Dan (30:11)
Mm-hmm.

Shane (30:12)
that screen is often informed by a bunch of other people's work, including user research. So yeah, it's true. And the other thing is certainly true, that there is no taste with AI. It's not going to meet your expectations because you still have to articulate your expectations.

Dan (30:33)
Right,

doesn't know good from bad. It doesn't know what's gonna work for your users or not. It has no judgment in that sense.

Shane (30:41)
But yeah, the role of research, I think, is to inform that and it will continue to be that. I think the methods are certainly going to change. ⁓

Dan (30:51)
Mm-hmm.

How do you see the methods changing?

Shane (30:54)
Well, there's a bunch of interesting things on that topic. I think one of the things that's kind of interesting is that paradoxically qualitative research is becoming much more important. And I say that because there's often been maybe a perception that survey-based or quantitative methods have been like kind of the gold standard within research, right? Like absolute truth comes from

Dan (31:05)
Everyone.

Shane (31:20)
data scientists. And I think what we're learning more and more is that,

Maybe there is no singular best approach for any given situation because what we're all trying to understand is what's in other people's heads. And that's not something an LLM can tell you. It's not even something behavior can tell you. It's certainly not something a survey can tell you. What is often the best medium for understanding other human beings is to have a conversation with them. And the fact that LLMs can mimic

Dan (31:35)
Mm-hmm.

Shane (31:51)
these types of conversations doesn't mean that it's meaningful in the same way,

Dan (31:56)
What do you think about? The kind of qualitative at scale where you have AI is doing the moderation of a script that a researcher writes do you think that that's gotten value in it? Because I know that was kind of always the thing about qualitative research was it's like, well, they only talk to six people. And now

In theory, with AI help talk to a thousand people, does that give qualitative more, more juice, more, more.

validation.

Shane (32:34)
think this is what I'm saying is if you think about all the other research methodologies that are not grounded in a conversation, they're very thin perspectives on the human experience, right? What we lose when we only talk to six people is understanding representativeness within a population. But what we gain is depth and context and understanding. And I think that's where the premium is placed now is understanding context.

Dan (32:45)
Okay.

mm-hmm

Shane (33:03)
So we have seen a bunch of tools pop up over the past year, two years now of AI moderated interviews. I will say I was extremely skeptical about them initially. And I think a lot of that is due to me as a researcher viewing my special sauce as my ability to extract information from people. And I've got a bunch of tricks for doing that that I've developed over the years.

Dan (33:27)
Right,

right.

Shane (33:29)
But.

these moderated tools are just as good, if not better, because they're more consistent. And I think that was certainly shocking to me. I think the other thing that was shocking to me is I had an assumption that people aren't going to be as open or they won't share as freely because they're not communicating the reality to another person. They're communicating it to a screen. In reality, people treat them like confessional boots, going much deeper in some situations, freely talking.

Dan (33:47)
Right.

Shane (33:56)
because there is no like social regulation in the situation. They're just freely responding to questions.

Dan (34:00)
Mmm.

was wondering about that because when I've gone out and done user research, I know that people are holding back because they don't want to say something that would be like, pooping all over my design or something like that. And they don't have that filter with the AI, it sounds like.

Shane (34:23)
Exactly. So I don't think of them as a replacement for qualitative research, like one-on-one interviews. I do see them increasingly as a replacement for surveys, Because they have the scale of surveys, they allow for qualitative data collection at the scale of a survey. The problem becomes, how do you analyze all that? How do you verify that any given insight out of it

Are you going to read 700 transcripts of a 30 minute conversation? Like, no. And this is often how research worked in the past. A researcher has eight conversations. They have a tacit understanding of what was discussed and where roughly people fall within a topical distribution. You don't have that if you just click send and everybody.

Dan (34:52)
Right.

Shane (35:12)
is talking to a computer screen. So you don't have this base level understanding. And then you look at data and you're like, I can't do anything with this. Let me get an LLM to summarize it. This is where research is kind of breaking down is the, it goes from collecting insight about people to knowledge about the world. Like it's that data to knowledge gap still, even, even though these tools enable like,

Dan (35:31)
Hmm.

Shane (35:37)
a thousand concurrent interviews,

Dan (35:41)
Right.

it's not the it's not the interviews themselves that are important. It's the it's the knowledge and insights you extract out of them. They're so important. And if you turn that over to LLM's. That.

Feels like that's missing something to me.

Shane (35:59)
Totally. And I think where a lot of these companies, they'll make tools to help researchers synthesize. And the problem is, is that it outputs a believable result. And believability and accuracy are two very different things in relation to AI. And so what do you anchor on then? Do we just anchor on the believability that this thing is telling me everything that I should know?

Dan (36:10)
Right.

Shane (36:24)
No, I think it still requires a high level of verification by researchers. We still need to show evidence.

Dan (36:32)
what's your hot take on these AI-generated personas that people seem to be hot for these days? Where it's like, we don't have to go out and talk to people. We have this persona, you can just interview them,

Is there any value to them? Are they better than nothing?

Shane (36:55)
I think in a lot of cases, synthetic data in general is a very nascent category. There are a number of companies that are starting to offer synthetic data specifically for product building and testing, evaluating ads and marketing copy. it's still extremely early. ⁓ A lot of the approaches that are being used to generate synthetic data.

Dan (37:14)
Mm-hmm.

Shane (37:19)
There's a variety of approaches.

I think they're useful if you really just want to understand what the average or stereotype is of any given segment. ⁓ Because the thing to keep in mind with how LLMs work is that they are trained on everything that has come before. Often though, ⁓ in research, our goal isn't just to understand something that is already known about the world. We want to learn new things about the

Dan (37:28)
Mmm, mm-hmm.

Shane (37:48)
whether it's in reaction to a change that happened in a market or we launched a new product or any number of things. So when you're talking with one of these personas, you should just know that it is a very watered down average or, you know, it is a human effigy that converges on the mean. And that's all you're ever going to learn. there are a number of

providers now who are like ⁓ figuring out ways of again generating response diversity that looks like how responses would fall within a survey question distribution. But it's still, it resembles human activity. It doesn't actually replace it. What's, what is becoming, and there's a lot of research on ⁓ simulations in general, but

Dan (38:24)
Right.

Shane (38:38)
As an approach, simulations, I think, are a more authentic and empirically grounded way of using LLMs to simulate individuals because you're interviewing a person for like an hour and you're simulating how that person would respond to a survey. And if you do this with enough people, you can get actual population level variance, but it's still anchored in

data and context collected from a real individual. So a variety of approaches, purely synthetic approaches, I feel are useful for, we just need signal kind of use cases, but in no way are they actually threatening researchers. In fact, what I just described, this simulating individuals actually makes researchers more valuable, Because we are the ones

who are going to extract and curate that context about that individual.

Dan (39:30)
Where do you see the value of user research happening in this new AI world? What are the places where user research is uniquely valuable?

Shane (39:43)
So if I was to reduce what user research is in the age of AI, I would say that we do two things. We extract context and we curate context. those are the two things we do. What is happening right now is organizations are adopting new tools to house all research in one place, right? And so the idea is like we're

migrating context to agentic AI systems. I think in the future, what it means is we are the ones doing that curation, also orchestrating the context collection end of things. And I think the ways that we're going to collect context are also going to change as well. AI moderated interviews is one example, but always on surveying,

is another popular example, In general, think the role of researchers also is going to be shifting away from like one researcher works with one feature team and they do a bunch of usability tests and predictable iterations. Instead, the role of researcher is going to be embedded in like a customer alpha or something where

teams are building at the velocity they build at, but researchers are adding value just by hosting the development context. I don't know what I'm describing here, but it's like our roles are changing from project-based work to like community managers in a lot of ways.

Dan (41:07)
I think all of us in this space are trying to figure out where, our roles are now, you know, given that, there's new technology that is giving us new tools and that's changing new processes. Like, and so we had to figure out all those things.

all at once and I think that's what's so hard for anyone in this area.

Shane (41:31)
exactly. And I think that's where research is probably struggling the most right now is to try to figure out if workflows in design and the relationship between design, engineering, and product are shifting and moving, where do we fit in? And so you see a lot of researchers posting on LinkedIn about using Figma Make to make a prototype, right? Rather than talking to a group of people for 20 minutes about

⁓ what problems need to be solved. Just show an example of a solution, But as design iterations speed up and speed up and go faster and faster, research can't keep up with that. How do we continue to add value when the value that we think we provide isn't necessarily valuable because speed is the thing that matters most? And so, you know, like I don't think it's a matter of like,

talking to people faster. We're not going to talk to people faster, but I think we are going to devise new ways of allowing teams to check or validate their work throughout the process that don't require a high level of human coordination project management. There will be systems where people can

Dan (42:25)
auctioneers or something? Yeah, like

Shane (42:47)
understand what are the usability problems that I'm introducing with this change that I've made in the system, or how is the button placement going to impact a click through rate. These things can be modeled now and it will be easier to model them more in the future, which means research, our value is upfront, helping define the problem space. And on the backend, validating what has been built and making sure that it,

Dan (42:52)
Mm-hmm.

Shane (43:13)
actually does the thing that we think it does.

Dan (43:17)
I wanted to talk to you about

agents a little bit. One of the things that I've been thinking about working with and designing agents is the role of user research in understanding the processes that are out in the world that we are trying to add AI into. And I feel like this is something that has been completely overlooked and people are just like, well, we'll just throw an agent at this, push the button.

And it'll just go do this thing without ever understanding the process that it is replacing and the human decisions that are really important that have to take place during that process.

It's not human in the loop. It's AI in the loop. Where are you keeping the humans in and where are the moments where it's good to inject AI?

Shane (44:04)
you really need to understand what is the intention of what the agent is supposed to be doing? Because the only way to really evaluate quality here is if you set, evaluation criteria against that known process. If I have an agent that is supposed to make me an ice cream sundae, but I don't myself know what an ice cream sundae is.

Dan (44:18)
Mmm.

Shane (44:26)
Anything the agent creates will then pass a test of Ice Cream Sunday, So, if you don't understand what the workflow for the agent ⁓ should be or how it should be ⁓ processing information, it's going to do whatever it does out of the box, which very rarely is what you expect it to like, the foundation for building

AI products are evals. so without understanding what it's supposed to do, you can't even do the foundation of testing the system.

Dan (44:58)
Right.

Are you the researchers really setting up those evals and metrics at the beginning with, with... No?

Shane (45:10)
Um, user researchers aren't heavily involved in that process from what I've seen, as more traditional tech teams are now working with AI because they have a remit to make their feature, uh, AI powered. We're kind of having to learn this as one big crash course, there's no set processes for conducting an evaluation of an LLM's out.

We're all kind of learning best practices as we go. But there has been a huge push for more people, designers, data scientists, user researchers, to become much more involved in the design and conducting of these evaluations because they really benefit from an interdisciplinary perspective. So if I'm an engineer,

I don't necessarily have the breadth of understanding of a design system or human cognition. And if what we're trying to

Dan (46:04)
Right or

user goals. All those kinds of things. And if you're brought in too late, engineering and data science have taken off and have just been like, well, we've put all these objectives into the model. And it's like, well, these aren't even the right objectives.

Shane (46:20)
Yep.

So yeah, I do think, it's like untapped opportunity right now. There's a couple of tools that have started putting out like, UIs for doing evals and systems for like building eval processes and stuff. but it's, it's not mainstream and evals are certainly mostly the purview of ML researchers still,

Dan (46:42)
you think people will start hiring more user researchers? You think think we hit the nadir and it'll be going back up again.

Shane (46:55)
I do think is true is artificial intelligence will result in more software, not less. Artificial intelligence will always be inherently limited by the context it has on the world. And so our job is to really bring new context into these models. And so in a way, until you can completely replicate my experience as a human being, research will be required.

Dan (47:22)
Well, thank you. Thank you, Shane. This has been great. Appreciate your coming on to the AI and Design podcast. You're our first special guest. So thank you for doing that.

Shane (47:26)
Great one.

Cool, Dan. Thank you so much.