< Back

Now Playing

Jeff Gelfuso - Qualtrics - TAB 1

Sorry.

So yeah, I don't know, this is, it's difficult to ask questions to someone that's like, oh, such a, the whole background of like, cold tricks.

Like, you guys are the experts in this.

Yeah, no, no, not at all.

Go for it.

No wrong right way to do it.

Yeah.

So one of the things that we just, there's kind of these questions we're asking and the big one that we're, asking everyone is like when you're moving, when you're looking at kind of moving forward with voice AI,

if there's anything that you could kind of wave like a magic wand at and just make it better or you

would just wave it at that.

Is there anything?

Yeah, so we're talking about this a lot and actually, Aaron and Stuart were We're here just, I don't know, a month or two months ago, whenever that was.

We were showing them some of the things that we're doing on the innovation side of trying to move a lot of the old school survey-based type responses to things that are going to be more natural than that.

Voice is a great example of that.

One of the challenges

in doing so

right now is latency and cost.

So

and what Stuart was showing me, which I think you guys are trying to solve, I just haven't spent enough time on it myself to understand is as a global company, obviously, I think the language picker part, so to say, it has gotten easier.

The latency, depending on where that capability is served from, in which data center, from in which location, right, globally

becomes harder because then it's at least that I know, I don't know of one specific partner that you can pick that's going to be great at all three of those things today.

I think what I understood a little bit more, and I didn't know before talking to Stuart about this more deeply, is you might have to have multiple providers in different regions to be able to make sure you've got both the right languages, less so now, but the right amount of capacity and latency solved.

And then of course there's the cost to that.

And some of those are much higher cost and some of those are lower cost.

And so those are some of the challenges that if I was to wave a wand today, I'd be like, boy, I don't want to have to think about that at all.

I would want to think about it like a hyperscaler where I'm like, AWS is everywhere.

We pick them based on the capabilities they provide, the cost they provide, the latency dance, and I could make a decision.

That would be ideal.

I don't think we're there yet from what I understand from a voice perspective.

Maybe we are.

Maybe you guys would know better than me.

Maybe we're making advances there faster than I'm even aware of, but that seemed to be some of the challenges balancing those things.

And

if

we were able to

just give you that, or if you have that, how would it change your life?

Well, we have this belief

and it's backed by some of our data in moving from more traditional, I'm going to call them surveys, but I'll say to all of you, Just think about them as a listening engine because we trigger them in different ways, right?

There's you're on a commerce website and you're trying something, you're trying something, you're trying something, and it's like not working and you're getting frustrated.

We can sense that frustration and pop you a survey, right?

That's an example of it.

I just got one this morning in Spotify and the app that was like, how's the listening experience?

Take the survey.

So there's lots of different ways that we can invoke that.

Where I would love to see this go and what it could enable us to do is be more dynamic in the delivery of the type of listening you were able to do based on a series of factors.

So let me give you my Spotify example.

Let's say I'm listening and instead of it being something that I get that popped up in the app and I have to actually tap it and then I'm going to, because it's Qualtrics, they're our customer.

I'm going to have to type into, like most of the time when I listen to Spotify, I'm on the car or on the walk.

Like if I got that survey and it just knew that, oh, this is a

music based or a audio based thing and it said, tell us what you think.

And I was like, great.

And it just talked to me and I did it all via voice.

I'd probably be much more likely to do that.

Survey like that and have a better experience in doing that where maybe if I was using

Jira, I don't know, and they said, take a certain, like, okay, great.

Would I stop?

I'm on my laptop, I'm doing work.

It's between 9:00 to 5:00.

Would I stop and do a voice-based one?

Maybe.

But I might just be able to do it like easy text.

Or voice to text or something like that.

So the context awareness

of

the options that I would want to have, I think will become more and more important and more differentiated.

You can imagine the different verticals like quick serve restaurants, retail,

Hospitality.

Some of these are really cool now.

It's not after the fact you go to a Marriott, you have a great stay, the minute you check in, you get a text, right?

And it's now not after the fact, it's in the moment.

Like, that'd be way cooler if I could just be like, hey, yeah, the champagne was awesome.

I don't have a tea time booked.

I don't like whatever.

Like, how do I use the pool or reserve a cabana?

Like, you're now having this kind of in the moment interaction.

If I've got to type all those things that that gets more cumbersome, then there's going to be some that have more regulatory

compliance that you have to have that may be limit.

So it's a long winded way of saying like the best of the best would be when the spectrum of those delivery and modalities is flexible.

Yeah.

And is there anything about making it flexible

that you would wave the magic wand out like specifically within the kind of like making it flexible?

I know you mentioned.

Yeah, probably like modality because like 11 Labs right now can do if you start speaking in Spanish and they've done it, it'll switch over and change from English to Spanish.

It was kind of magical and feels pretty cool.

And back and forth, right?

But

I would want to have an understanding of the type of customer in the type of industry and the type of listening they would want to do and have some dynamic modality choices.

It sounds really nerdy, but I know you know what I mean by that.

So Spotify's all mobile.

It's all an audio thing.

Don't ever give me a text-based survey on Spotify.

That's the same stupid.

So if I could do that and I could provide that, our customers could select those kinds.

I just got my burger, I'm walking out of the restaurant, I go to grab a bite and the thing's cold.

Am I going to go whip out a laptop or get on my phone and find the contact that no, I should be able to be like,

Hey, I got this.

How was your...

Hey, it sucked.

The burger was cold.

Whatever.

Yeah, yeah, yeah.

Less friction.

Yeah.

Right, that's right.

Yeah.

And is there anything about, like, is there anything that makes this more valuable now than it was like a year ago or two years ago or even longer?

AI in general, like, they're just, the advances now are so fast, they're changing so fast.

I don't know if there's a new law, but like Moore's law, like you just now the cost part of it just like moving to the cloud was before, like pretty soon that's gonna be a race to the bottom or cents, not dollars.

And then the cost question will be less of an important

but it's still a near-term thing because for somebody like Qualtrics, We've moved from, with the exception of one business, but we've moved from licensing per seat to interactions, right, and how we charge.

And so what we want and what's valuable to brands or customers is increased number of interactions.

So the more surveys that are responded to and completed, the more call centers contacts that are deflected, the more like those are all interactions.

We get paid on like you buy 100,000 interactions, use 90, great, use 120, we make more, we need to charge you more, use 30, you want your money back, right?

Like so there's some threshold there where we're trying to manage that.

But what's great about it is that the business gets more data, the customer gets more data, which is more valuable to them because they can use AI to analyze it and target specific things.

We get more interactions, which we can charge more for.

So it's beneficial to both the customer and to us.

AI has changed that radically.

And even in our most recent conversational surveys, you can say,

How was the game?

Game was great, but the food was bleh.

What do you mean it was bleh?

It can detect that it was, and you say, well, they didn't have the menu items.

What other items would you have liked?

I'm gluten free, so I would have loved better choices than just chicken fingers and French fries.

Now you have all this great data that the business says, and what we've actually seen in that AI implementation of a survey, is it's more natural, people are having more of a conversation, they're getting more valuable data because now you can target those specific things.

We're able to increase the number of interactions we have and we actually saw that even responses went up because the person felt more heard versus just you got the one thing and you moved on to the next thing and they're like, okay, I'm done with this, like stupid, right?

So that those advances crazy, crazy good in the last year, and I think we'll continue to be.

Yeah.

Yeah.

It's amazing.

It's cool hearing about it, like, at the scale that you guys operate.

Yeah.

Is there, is there a number two thing that you would wave a magic wand at?

I don't know if this Lots of number two things, probably.

But they probably have more to do with organizational dynamics, culture, speed of building, things that Aaron knows so well that we've dealt with and deal with in large organizations that probably have less to do with the technology or the solution.

Specific to voice and AI that way.

Yeah, that makes sense.

Do you just mean like compliance and red tape and procurement and all that kind of like, is that what you're talking about, Jeff?

No, I think, I mean, sure, those things eat away at your soul, right?

But

I mean more like some of the new tools and things we're exploring

Subframe lovable cursor, when certain, like just they just help you build so much faster.

Aaron's using Claude code and building like,

like never thought Aaron would be doing it.

Never thought.

Never thought, right?

So when I think my magic wand is, when I say it's more the cultural things, the tools I just had a town hall this morning where we were talking about, because now I'm leading all the product and all of experience, and I'm bringing those two organizations together like I had previously when I worked with Aaron.

More and more of that now is like, well, but a product manager can build a prototype.

What does that mean for me as a designer?

Designer can write a great strategy document.

What does that mean to me?

The tools now are blurring the lines of the roles.

A little bit, we'll all figure that out.

When you're in a small company like yours, someone like Erin wears 10 hats and now can do things that she never could do before.

That's amazing.

You're not limited by the cultural

inertia of that because Aiden, Jack, Erin, Stuart sit together and go, well, we don't want to do it.

Erin should do it.

And Erin figures it out, right?

I have to go, what does that mean for HR?

Is our roles gonna change to compensation to where we recruit from?

Like all that shit.

So like, if I could just focus on how do we build better and wave my wand on that second thing,

I don't think you guys are set up to solve that.

But

that is gonna be huge for big companies over the next several years.

Does.

Any of that more make voice AI specifically more difficult to bring in or are there any specific challenges around voice AI?

None more than I think what we talked about, which is kind of the scale, latency, cost, stuff like that.

When you demo an experience, and people do it and it's voice based, it's so natural.

It blows people away.

It just, it totally blows them away because, and the things that you would even, my example I gave you about our conversational, that's still text based or, and you know, you're typing,

even though we're seeing more turns and greater detail and responses, there's still only a level to where that will go because you're just like, I'm not going to type this out, this whole entire thing.

But if I was just going, oh yeah, my Spotify experience, it's great, I love all the music, I wish they had these artists, tell me more about that.

You're just having a cut, you're going to say 10 times more because it just feels natural.

And when people actually take it and we've built both, they go, that didn't even feel like taking a survey.

And that's a huge, you don't have that without voice, that's a huge deal.

Super interesting.

Yeah.

Was there any of the voice AI providers that you've looked at and just love to know how you've been thinking.

About.

Evaluating them and that sort of thing?

Yeah, if you guys want to do a follow up, Dave on my team who Aaron and Stuart Matt is way more versed in this stuff than me and he's been building the prototype that we showed Aaron and Stuart was from subframe to Supabase to 11 Labs to cursor or some version of all those together to build this dynamic, AI driven.

And what's really cool about the way he did it was you can bring in things like Braden guidelines and knowledge-based articles, and you can adjust tone, all the things that you would expect, right?

Pick a second language, do these type of things.

But what's pretty game changing about it is how AI can ask in real time

based on the variation of your answers.

And that's our unique part to it that we obviously will continue to build out on and understand sentiment and more like those emotions than the specific just things that are more black and white that you don't get.

Eventually, I think voice could help even with tone and inflection.

Like, could you like we have it right now where if it detected words that said, hey, you know, I'm super busy.

I got to go.

It knows to skip all, oh, I got to go to the very last question and see if I get them to answer that.

Right.

So.

So there's some of those things, but it doesn't know if I'm like, like, it probably could hear that, but it.

Or maybe it was like, like, could it, could it?

Or if I was like, this is so joyful and wonderful.

Like, could it understand?

Asking.

Right.

You don't get that with text.

You don't get that with, but you could with voice.

So that could be like imagine a patient experience where there was a high level of anxiety maybe.

And could you detect that?

And then, or maybe it was after some kind of major event and you could detect more of that emotion and tone, that could be really interesting.

Yeah, that'll be so amazing.

Yeah, I think those were all of the questions that I had, but I don't know if Aaron, Aiden have questions.

Yeah, and I was going to offer if we want to set up something with Dave, kind of demo some of that and just show you guys.

I think it was pretty illuminating for Stuart and Aaron when we showed them, but happy to do that as well.

Yeah, that's so I think that'd be great, Jeff.

Thank you.

I think we'll take you up on that.

And like just hearing you talk about that, I think, you know, these are the most interesting places for voice, something I was talking to Damian about yesterday and is a real area of interest for us is like, like you said, kind of multimodality.

It's like just meet someone where they are.

Like where is voice actually good versus where is text good?

And there's been a couple of products we've played with and then actually a lot of conversations with people in healthcare building agents.

This has come up where like data capture when it's a name or insurance number, you need 100% accuracy on that stuff.

And so voice actually isn't so good for some of that because there's agents that do outbound calling and they get someone's name wrong and it's like the whole thing's frustrating for everyone and the results useless.

Right, right.

Whereas obviously the areas, so places like that, it's like can you integrate SMS capture to the first step.

So we know 100% who you are before we have an agent speak to you.

But then as you said, areas where you actually want someone to just like tell you their thoughts and get more from them.

Voice is great.

It's way better than text for a lot of the stuff you guys are doing.

So yeah, I could imagine with like medication names.

Oh my god.

Yeah, exactly.

Right.

And it's like the user might not even know the name.

Right.

You can think of all these edge cases, but I think where these systems get really powerful is like, okay, where they nail which modality makes sense, how do we actually meet you in the place where it's low friction for you to get us more from you?

And then as I said, if the voice system can actually, the other thing that voice has that is tone and emotion and all these things, which you don't get that at all from text.

So as you're thinking about, I think that's where this stuff gets really powerful and probably more fun for the user, which means more engagement and more data.

So I'll give you a great example.

We're doing some of that in our surveys for customers like Marriott who have multiple brands.

So for somebody like they own the St.

Regis, it's more, good evening, Mrs.

Sullivan.

How was your dinner tonight?

Where at the Moxie, it's like, yo, what's up?

And you could literally change it by brand, I'm being extreme about it, but you get the idea.

That's super powerful for multi-brand companies.

And it's only kind of happening in text right now.

It could also be, and the voice part of it is only more like language or

regional inflection accent.

So you get the British accent for the use.

But imagine if it could be like,

Hip hop tone language.

I don't know.

You're gonna meet the user where they are, right?

We heard literally yesterday, like people in Texas want to hear on the phone someone with a Texan accent calling them to ask them questions like conversion rates higher.

Yes.

You think about that across every state in every country.

Right, right.

But you could even think about it generationally as well, where there are certain words, our kids, don't say anymore or wouldn't say or wouldn't even, it wouldn't, you know, even impact it.

And you could change that knowing some of that information and the demographics on the fly would be cool.

I was just thinking what you said about the burgers, Jeff, because you guys have Kentucky Fried Chicken as a customer, isn't that right?

Like imagine someone's got chicken all over their hands and you do a survey, you probably should voice it.

Well, what we're not doing yet in those specific regions, like right now I think some of those customers do a QR code scan on a receipt.

First of all, whoever keeps their receipt for a $6 meal, right?

Second of all, probably if you kept that receipt and then you were walking out and you had hot sauce on your hands and you hated the chicken, are you going to really pull that out, scan it with your phone, If you do that, all that kind of effort, it's probably because you're going to give us a negative review.

Not, this is the best damn chicken I've ever had.

Like maybe, but so what we're not doing a good job yet is,

I would say the good examples are hospitality where you check in and the bigger companies send you a text and now you start having a dialogue.

So you have a during stay experience.

Not only a post day experience.

That's really valuable.

We haven't done a good job of understanding that after a hair appointment or a rest.

I was in a restaurant or like, could we could you have some kind of geofencing or some kind of way where, hey, I was just in this place.

I left that place.

Bing!

Automatically pops up on my phone and says, how is the chicken?

Oh, okay.

That's pretty cool.

Now I might say, oh, tasty.

But if you have to do all those jumps and hoops, it's probably because you just really are pissed off and you're going to do it.

Otherwise, you're pretty much not giving any feedback at all.

Retribution feedback.

Yeah.

So I wonder about location a lot in our industries, whether that was a hospital, a doctor appointment, a restaurant visit.

What other new capabilities will come up that then you could combine with different modalities and then different ways to have those text or voice kind of entries.

Who knows?

Maybe eventually five stars can just be like Zoom says, how was it?

I go, and I gave it five stars.

I don't know.

That could be cool.

So cool.

Jeff, I realized we're at time, so I don't want to make you late for anything, but I just want to say thank you so much.

Like, it's so, so amazingly helpful.

And, yeah, it's, like, so cool because all the answers that you share are, like, so meaningful and so, like, have a huge impact because of the scale you guys are at.

So thank you so much for your time.

Absolutely.

Great to meet, meet you guys.

And, um, Aaron, if you want to follow up, I could.

I could see if Dave could do the demo, and then if there's any.

Any other thoughts or questions or things like that,

let me know.

Cool.

Yeah, I'll reach out to Gabrielle then.

Yeah, that would be great.

Okay, awesome.

Have a great rest of your day.

Thanks.

I see you.

Bye.

Bye.