Nik (00:05)
Welcome to AI and Design, where we explore how artificial intelligence is reshaping the world of design. I'm Nik Martelaro

Dan (00:10)
And I'm Dan Saffer, and we're faculty at Carnegie Mellon's Human-Computer Interaction Institute. Each week, we break down the latest AI developments, dive deep into topics that matter to designers, and talk with fascinating guests who are right at the intersection of these fields.

Nik (00:27)
Whether you're a designer working with AI or an AI practitioner interested in design, we're glad you're here.

Dan (00:33)
And I know you're all going to be shocked at this, but on today's episode, we're going to be discussing the launch of Anthropic's Claude Design.

free to.

Nik (00:41)
And I spent

last week in Barcelona attending HCI's biggest conference, CHI, and I'll tell you what I saw. I've also got some mini interviews with researchers working on the cutting edge of AI and design research.

Dan (00:52)
I'm really looking forward to hearing the report. I've deliberately kept away from all the CHI news so I could hear it right from you. But first we have to talk about the elephant in the coal mine here. So I'm out last Friday, living my best life. All of a sudden my phone starts blowing up. Text, Slack, signal, Discord messages. Have you seen it?

Have you seen it? Have you tried it? It launched. And of course I'm talking about Claude design. This entire weekend on LinkedIn was insufferable. My feed was nothing but thought pieces about Claude design. People trying it, people liking it, people hating it, people weighing pros and cons, lots of hand wringing. Nik, have you tried it?

Nik (01:42)
Yeah, so I tried it and I actually did a couple of things with it. I did one task where I wanted to create like a new lab website. And one of the things that I saw in the demo, which I really liked, was the ability to generate options. So you can say something like, give me six layout options for this. The other thing that I tried was doing it in sort of sketchy mode, wireframe mode, as opposed to high fidelity mode.

You know, as someone who actually still even today likes to work mostly by hand, I appreciate the sort of sketchy way of looking at things. I think that working in those lower fidelity media are useful because you're thinking more about content layout, information architecture, as opposed to pure say aesthetics. And yeah, honestly, it did a pretty good job. Now I asked for something that also could be done very well with like a WordPress template or

today, you know, like a GitHub Jekyll template for a lab website. But I really liked the fact that I could ask for options and it generated them. The other thing that I liked a lot, and this one I was like, this was super cool, was the tweak mode. So the tweak settings where you could say, hey, I wanna tweak color,

I, for example, I had it so I could tweak primary colors, fonts, font sizing, spacing of say carousel or card interfaces. And it was really nice to be able to like just click those things and then see the differences and see how they would look in the full wireframe without having to say regenerate something, especially when oftentimes that's what

Dan (02:58)
Yeah.

Nik (03:12)
found was really challenging working with say, Claude Code and generating a prototype of something and being like, wait, but I want to just adjust this a little bit. Like having the tweak mode was really, really nice. The other thing that I tried was a presentation because I just came back from CHI. actually, I had to give a talk while I was there. So I made my presentation myself. I did it in keynote to like I've always done, but I said, hey, you know, how well would it do? And I gave it

the outline for my slides and then said, hey, I like to have black slides with sans serif font, something clear and readable. And then I drew a sketch. This is the thing that I loved being able to do was drawing the sketch and say, here's kind of how I like to lay my slides out. Little title at the top, an assertion on one side, and then a square, like put a figure or an image over here. And I said, you know, look at that. And it was a, just.

really garbage sketch but one can imagine you know sort of what I did and it did that. No, arguably that's a pretty standard slide layout. But I didn't describe it. I didn't describe what that looked like. I did just say here's the sketch. That was pretty good. I will say everything I got looked good and it looked good in the way that a lot of

things look good today. It felt like it was missing something, but it wasn't totally bad.

Like, there's something that I'm missing there, but I'm not totally sure what it is. Dan, have you tried it?

Dan (04:35)
I have tried it and yeah, I was impressed by it. I definitely felt like it was the most robust of these kind of tools that we've tried recently, And I went a little deeper with it. My daughter had a pretty good startup idea and it's

Fairly complex. not going to give it away, but it's got calendaring. It's got some social aspects. It's got some validation. It's got some search. and I plugged in a pretty decent size prompt and because it's summertime theme again, not giving anything away. I mean

who knows if this will ever go anywhere, but it's her idea, so not mine, but I was like, what could I plug into this thing and try? and let it run, and the first outcome was pretty good. Like the style was okay, and the general layout seemed okay, yeah, and then I did a couple tweaks using the tweak mode, just like you said, that was.

That was awesome. And then I was like, well, it's kind of missing this piece. This doesn't make that much sense. And as I start to kind of push into it, there was a lot of things that were like, well, this is there because I thought it should be there. And in some cases, that idea was great. Like it had thought of a couple

add-ons that I hadn't and I'm like, well, that's a very clever idea. I'll keep that for sure. But other times it was like, here's a notification. And I'm like, okay, well, what, what is it? What am I being notified by? And it was a completely dead icon for notifications. And it's like, well, what are we notifying here? I don't know. So there was a lot of kind of dead ends like that. And

because it was in a context that I actually know and I'm like, well, It didn't reflect what the end user group cared about the absolute most. And so I had to add that in as another prompt and put that in there. And I'm like, well, this is where

having some kind of contextual knowledge through user research or through lived experience really mattered because I was quickly able to say, well, this isn't just not as helpful as it could be. And so really having that and digging into that was really interesting and it added it and added it great, but I had to know that it wasn't there.

I Knew what it didn't know and so that was helpful. But if I didn't know what I didn't know Then I could then I couldn't have helped shaped it

Nik (07:21)
Yeah, I should say I think that for me, the reason I did the slide design was because actually just a week before my talk, was like, well, I don't know, can I get Claude to make my PowerPoint slides for me? Like, hey, I got this outline and stuff like that. And I tried it and it creates a PowerPoint. Oh, and it was bad. Like it was a black background with white text, but like it couldn't even get the...

Dan (07:39)
Right.

Nik (07:43)
positioning of the titles consistent, which I was like, I don't even understand how you do that. Like it's a template like PowerPoint, like it should be every slide. Like what? This was a huge improvement. I should say if you're out there and you've tried, for example, like Claude Cowork to create, say PDF or PowerPoint type assets before and have been really underwhelmed, this was a...

huge improvement. Like this feels in some ways like the step change in regards to like the same way we had with the coding agents. ⁓ This feels like the step change moment.

Dan (08:10)
Right.

Yeah, I mean, regular Claude can't format a word document with any consistency, much less a slide deck. And yeah, this, this does feel like a big jump. I'm giving a talk on Wednesday. And so I'm sad that I'm like, maybe I should try and put it in and see how, see how it goes because Claude helped me do the outline of the talk. So I'm like, well,

Can I just port that over? maybe I'll give that a try. I mean, one of the many threads of discourse around this was like, is this the end of Figma? And I'm not sold on that yet. I do think that Figma has a lot of

brand loyalty. It's got deployment and thousands of workflows. It's got that, SAS buy-in already. It's beloved by the design community. So I'm not ready to throw in the, the figma towel yet, but, on the, on the flip side, I heard another statistic that

was pretty surprising to me and maybe it's surprising to you that only a third of Figma's users classify themselves as designers. The other two thirds, not designers, a third of which are kind of other, which I'm guessing is like marketing and product management. And another third are developers. And for...

particularly the PMs and especially the developers who are probably already in Claude and hardcore Claude users. I'm guessing that will be a big defection from there to this tool.

Nik (09:51)
Yeah, I was also really surprised to see that. guess I had thought it was higher, but maybe when I thought about it a little bit more, it makes sense because in many ways, Figma is the shared canvas for a lot of the design work, but a lot of the product work, right? It's like, have Figma, you've also got FigJam. And so this is sort of that shared visual board in which you're working and we're all team members.

Dan (10:13)
and they've got the stuff like export as PowerPoint

Nik (10:16)
so the Send to Canva was interesting to me because they have had a lot of reasonable partnership with Figma, right, getting MCP working, they've talked about that in other stories. The fact that there isn't a Send to Figma button here

I don't know what that means. I don't know if I want to read too much into it But it's interesting that they've decided to partner specifically with Canva. I wonder if the play here maybe their positioning Claude design as Like Canva like Canva users are often not professional designers, though. They can use it their professional designers are in figma but everyone else, you know my students who

need to just make a poster, right? They're in Canva because they're getting a good template with a little bit more design sense than like a PowerPoint template. And they're like, yeah, this works great. So it's interesting that that's where they position themselves on that export.

You can upload a FIG file. So you can upload a FIG.

Dan (11:08)
⁓ okay. So you can't. Yeah.

right. I mean, the other thing we didn't talk about is, when you first boot this thing up and you're like, Hey, I'm gonna make something it's like, okay, where's your design system? You got a design system? I'll take that. And that's super nice. Like, okay, well, yeah, sure. Take it. This is great.

Nik (11:23)
Yeah.

Dan (11:28)
because that's been one of the tripping points for a while now is getting those things synced up. So this thing does it like super easy. That's a big bonus.

Nik (11:39)
Yeah, I mean, one of the things that I saw as I was using it too was the fact that even if you don't have a design system, it starts to create one for you. And then that's what's allowing the tweaking and the editing, which is really nice. But it also got me thinking as I was using it, it does everything effectively in code. Like it's always writing code to do stuff, which makes sense. That's how you have to like render it. But it's always like, I got to write code.

Dan (11:48)
Right.

Nik (12:05)
Which I found interesting and I kind of have been wondering, is that limiting? Does that limit us?

Dan (12:11)
It's definitely token limiting. I saw a lot of people over the weekend being like, well, there goes my token limit for like two pages of this thing. So it is, it is token intensive Now, will we start seeing token maxing amongst designers now? Maybe, I don't know.

Nik (12:13)
you

Dan (12:30)
But you know thinking about this I mean The other thing I was thinking about so figma is a two thousand person company Anthropic is a twenty five hundred person company figma that two thousand person company is Figma is pretty much all they do. I mean they have you know figma make and you know and fig jam and some other little things but anthropic

How many people you think that they put on this out of the 2500? Maybe, maybe 20 maybe. So it's like, ooh, boy, that's some tough math there in the sasspocalypse realm.

Nik (13:11)
That's a pretty interesting point, and I hadn't really considered that, ⁓ the fact that they are able to kind of recreate these tools. And the thing here too is they're able to create them quickly, right? The level of capability, I mean, we said that when we talked about Google Stitch, right? I was so impressed at what...

likely a similarly small team at Google was able to produce already kind of knowing the answer. I mean, that's the thing, right? Figma has already gotten an answer in regards to like, this is what a good design tool, looks like for many, many people. And so, you know, what, what I say that these are wildly different or they, are they taking a completely different path? Maybe not, but at the same time, yeah, it is being done with much fewer people. They're being done a much more rapid.

timelines and the fact that they're integrated. mean, I'll say myself, I am using Claude a lot more for my work. Like the way in which they have just built co-work and built the integrations. I'm like, yep, it's starting to work and I'm starting to kind of get into a rhythm of like understanding how to use it. And so the more that these are actually it's even one of these things now when I have a tool that I can't connect, I'm starting to almost get like frustrated.

Dan (14:23)
Mm-hmm.

Nik (14:23)
Like, why

can't I just make these things talk to each other? Like, just want Claude to be able to talk to this tool and like do some stuff. Maybe not everything, but just some stuff.

Dan (14:31)
And I'll say the other thing about this is that Anthropic owns the models. They can build this with their latest and greatest model and then power it with the next greatest model, is what, Opus 4.7? I'm really bad about remembering all these names. 4.7. Meanwhile, you look at Figma Make.

And it's back at the 4.5. So they're like, figma make powered by anthropic using an older model. So even if you are using figma make, you're not getting as good a results because you're on an older model. And

they would have to buy a separate Claude version in order to build something like Claude design. like, I don't know, is Figma funding its own demise here?

Nik (15:25)
I don't know. I mean, I would say if there are folks out there who work at Figma and you're interested in talking to us, I mean, it'd be really interesting to hear, you know, from your perspective. If you can talk about it, I understand this is probably quite sensitive, all kinds of stuff, but I mean, it would be interesting to... Yeah.

Dan (15:40)
my God. Come on our show. Get fired. It'll be awesome. Yeah.

Nik (15:45)
Okay, yeah, I guess not. I'm sorry. Forgive me, folks. I'm an academic. We get to talk about

whatever we do for the most part all the time. Yeah, I'm not sure.

Dan (15:54)
But

yeah.

Well, I think what we should probably wrap this up with is that we are now almost two months exactly out from the big Figma Conference, config in June and

Boy, the heat is on for that to see what is going to happen with things like Figma make. Figma make they just announced last year at config and that was almost exactly a year ago and then everyone kind of got it in July of last year if I'm getting the timeline right. So boy have things changed in less than a year's time

So crazy times, crazy times. will be at, this is why we have a weekly show.

Nik (16:36)
This is why we have a weekly show.

Dan (16:39)
I will be at config and be doing a report there. So I'm looking forward to meeting folks and interviewing them for the pod there. And speaking of which, Nik you just got back from a big conference. So I'm glad you're back. And I want to hear about what you saw,

Nik (16:57)
Yeah, so I was in Barcelona at the ACM Conference on Human Factors in Computing, or CHI. it's our biggest HCI conference of the year. This year was basically maxed out at

I think 5,800 people, which is kind of wild for HCI. We're not like the AI conferences that are today, are, you know, like now I think they're even up at like tens of thousands of people. know, HCI is still not that big. And this was massive. And yes, there's a ton of stuff. First off, AI was everywhere. I think they had, I don't know how many sessions of AI, but you couldn't...

go to every room. It was literally impossible. So this conference, just to give people a sense, it's in a big convention center and there are something like 15 to 20 parallel sessions going on. So there's like 15 talks going on at any one time. And then you try to like pick one and then there's seven talks within that, within a session, within about an hour and a half. So everyone gets like 10 minutes. So I was there and yeah, there's a ton of AI stuff. And I was trying to find some of the coolest

new work that I could see in regards to AI and design. I'll say that unfortunately I couldn't see every talk. Actually, oftentimes there were rooms where they were so full, like it was like waiting in an amusement park. You had to wait in line outside the door for seats to open. And this sometimes didn't even happen. But yeah, I actually.

saw a number of really interesting talks, often from students. So this is cool. These are PhD students thinking about sort of the future of AI, the future of design, where could these things go? It's not just academics who are there. There are a lot of corporate research labs. For example, Apple was there. They actually had quite an interesting paper.

⁓ on improving user interface generation models from designer feedback. This was actually led by, I believe, ⁓ a student intern, Jason Wu, and then a number of other people from Apple, basically showing that the standard model of saying, I like this, I don't like this when things generate UI isn't really enough. And actually, it turns out that having a designer, like professional designers, actually give more critical feedback.

comments actually talk about and give their real sort of more, you know, in-depth knowledge and then providing that feedback to the models actually leads to much better improvement and reinforcement learning of those things. And on the one hand, I was like, yeah, that kind of makes sense, right? You actually ask a bunch of professionals to kind of like really critique and comment instead of just good or bad or I like this one better or I don't. But it was cool to see and I think one of the cool things about this paper is that

This is something they did with, you know, it's like 21 designers, 1500 design annotations. It sounds like a lot, but actually like that's totally doable. I think for a reasonable size company or over a long enough time, you could start commenting on stuff, collecting this and then right. Basically tuning your own designer reinforced models that really are taking in your expertise, your style.

Dan (19:48)
Hmm.

Nik (20:03)
the way you think, which I thought that was pretty cool. So that was like one paper.

Dan (20:06)
So who would use that? it for like individual designers to like make their own style or is it gonna be like, is that research that's a anthropic or Figma or Google is going to take for their own models?

Nik (20:22)
Yeah, I could see this in a couple of ways. I could see this working potentially for an individual. if you're trying to basically, actually, if you're using something like Cloud Design and in the same way in which you were developing, say, a skill file, like this is another way of basically saying, hey, generate stuff and then I'm going to add comments and you're going to learn from that. I could actually totally imagine something like an Anthropic or a Google, you know,

creating this as a feedback mechanism within their new design tools. I could also see it within Figma. Like, Figma Make could do this and say, add all your comments. We do that now. Make all your annotations. And then it starts learning the model better. And that actually could lead to much more improved.

generation and output ⁓ than just sort of like the bulk improvement over time. Like we have to wait for the companies to do that. Another. Yeah.

Dan (21:05)
Mm-hmm.

Actually, that's pretty interesting.

It's interesting that to use the comments that are already there because people are already commenting on these things. Now, how much they're really focused comments or how much they're contextual versus actual design comments is probably a question, but I mean, there's gotta be.

I don't know, millions and millions of comments that have been added to Figma files over the years.

Nik (21:42)
Right, and so in theory, right, if you could train on that, I mean, that could be a huge advantage. when Figma Make came out, it was its own step change to generating user interfaces with OpenAI, with anything else, with Google AI Studio, like a lot of, like I remember when my students started using it, they were like, yeah, I start all of my design work in Figma Make.

for layout and stuff because it's so much better. It just understands it better. Then I move over to Cloud Code. Now it's interesting to see, and I know we're going back to the original story, but it's interesting to see Anthropic catching up here, or I am potentially matching in that way. so who knows? For all we know, they're even utilizing methods like this. mean, this is really cool that Apple has done this research. They've documented it. They've showed it and showed how to do it.

And I think that these methods could be really quite useful, again, for maybe individuals, but potentially for the large companies, or just for your own company, basically, to develop your own internal systems to kind of improve stuff so that everything gets generated in a way that it meets your brand, your product.

Dan (22:46)
Mm-hmm.

Nik (22:47)
Yes, so this is one paper that really stuck out to me. But there's tons of work here. There's people who are applying AI to make it like a teammate. And I talked to some researchers who are doing that and have some ideas on that. There are people who are trying to capture designer knowledge. So very similar to this one, they're using other strategies, but they have like a agent that floats along in your interface. People are trying to figure out

what professionals are doing. They're interviewing professionals to basically try to see how they can take what those professionals do and apply it to AI workflows. There's so much going on and I'll say I am only catching a sliver of it.

with some of the talks that I saw that I enjoyed and sort of pulled people off to the side and said, hey, can I interview you? So yeah, for the rest of the episode, basically we've got a rapid fire set of interviews that I did with a number of different researchers who had work that I thought was relevant to professionals. And they'll talk a little bit about their work, talk about why it's important, and then where they think it's going to go next.

Dan (23:47)
All right, well, let's go to the videotape then and we'll be back with Nik's interviews from CHI

I hope you enjoyed those interviews from Barcelona and we'll be back next week with more AI and Design news. We'll see you then.