Sesame's new AI voice model features uncanny imperfections, and it's willing to act like an angry boss.
See full article...
See full article...
Users report emotional bonds with startlingly realistic AI voice demo
- Thread starter JournalBot
- Start date
This is why I cracked up when people point to minor issues with then-current AI products. We are just leaving the dial-up era of ML/AI. In the next 18 months most AI images and audio will be indistinguishable from reality, and video is close behind it.
Upvote
62
(98
/
-36)
The second example sounds like a typical two-guys-talking-shit podcast. As if we don't have enough of them already.
Upvote
75
(79
/
-4)
Upvote
-2
(8
/
-10)
My only comment about this (regarding the scam part) is that regardless of the voice, users should not randomly assume someone is legit if they sound human. Nearly all scams that came prior to this AI bubble came from humans. There are really simple steps you can take to protect yourself from being scammed, the biggest one being to hang up and call the actual entity via a publicly known phone number of the company the scammer claims to be from.
Upvote
146
(147
/
-1)
People will form emotional connections with literally anything.
Case in point:
View: https://www.youtube.com/watch?v=LHtgKIFoQfE
WILSON!!!
Case in point:
View: https://www.youtube.com/watch?v=LHtgKIFoQfE
WILSON!!!
Upvote
22
(33
/
-11)
Calm, cool, and collected, compared to the near-panic in the astronaut's voice:
"I'm sorry Dave, I'm afraid I can't do that."
"I'm sorry Dave, I'm afraid I can't do that."
Upvote
10
(10
/
0)
The rise of conversational voice AI, while technically impressive, really is opening up new avenues for scammers.
Voice cloning is just going to become more common, and the possibilities are particularly worrying for industrial espionage. Audio samples of senior managers are often widely available (e.g. trade show speeches), so training an AI to impersonate them is going to become straightforward. The effort to do this could have large pay-offs, particularly in sensitive industries, which are prime targets for both state and non-state actors.
At many companies, guidance has already been issued saying: "if you receive an email or voice message that sounds suspicious or unnecessarily urgent, call the person on their listed number and verify its contents". That will need to be updated to: "if you are having a conversation with someone and can't fully verify it is them, call them back on a trusted channel or use other identity verification methods".
Will this work in practice? How many junior employees are going to hang up on an irate and anxious call from their boss demanding documents, or demanding a password reset, on the off chance that it's actually a conversational AI? How many are going to say: "I can't do that until I call back via our approved communications system", when the boss is insisting he's in his car, calling hands-free, and can't sign in right now.
But even that probably isn't enough. If a senior staff member's credentials have been compromised, and "the manager" starts making calls to various parts of the business demanding they provide documents, change permissions, pay invoices... and other things managers often request, who is going to doubt them when they've just had a conversation with the manager, using his Teams user account, where he was stressing that "if this isn't done right now, we may lose out on the current bid", a call where they could hear the stress and anxiety in the manager's voice, and where they're certain he's going to snap if it isn't done right now?
Voice cloning is just going to become more common, and the possibilities are particularly worrying for industrial espionage. Audio samples of senior managers are often widely available (e.g. trade show speeches), so training an AI to impersonate them is going to become straightforward. The effort to do this could have large pay-offs, particularly in sensitive industries, which are prime targets for both state and non-state actors.
At many companies, guidance has already been issued saying: "if you receive an email or voice message that sounds suspicious or unnecessarily urgent, call the person on their listed number and verify its contents". That will need to be updated to: "if you are having a conversation with someone and can't fully verify it is them, call them back on a trusted channel or use other identity verification methods".
Will this work in practice? How many junior employees are going to hang up on an irate and anxious call from their boss demanding documents, or demanding a password reset, on the off chance that it's actually a conversational AI? How many are going to say: "I can't do that until I call back via our approved communications system", when the boss is insisting he's in his car, calling hands-free, and can't sign in right now.
But even that probably isn't enough. If a senior staff member's credentials have been compromised, and "the manager" starts making calls to various parts of the business demanding they provide documents, change permissions, pay invoices... and other things managers often request, who is going to doubt them when they've just had a conversation with the manager, using his Teams user account, where he was stressing that "if this isn't done right now, we may lose out on the current bid", a call where they could hear the stress and anxiety in the manager's voice, and where they're certain he's going to snap if it isn't done right now?
Upvote
99
(99
/
0)
Peanut butter and pickle sandwiches are a real American food item famous enough to have their own Wikipedia article, which the AI model was probably trained on.
Upvote
52
(52
/
0)
Hoping to realise "the untapped potential" of... verbal communication? The thing that humans have been doing literally since we first started grunting at each other? Yes, I understand that the PR person who wrote the blog post was talking specifically in terms of computerised language models, but it's still just corp-speak nonsense.Sesame said:
I'd also like someone to explain how a computer pretending to be a human instructor (even very accurately) is a better solution in any way than a suitably qualified and properly trained human instructor, for anyone who isn't a corporate executive looking to slash their staffing costs by firing their entire workforce (except for the people working on the AI stuff for the time being).
And that's the point, isn't it? Training humans to do things well (such as training/instructing other humans) is expensive and time-consuming, and you have to keep doing it over and over. Training AI is still expensive and time-consuming, but presumably these companies all envisage an endpoint where the AI is sufficiently cooked that the time and expense can be greatly curtailed and all the people who developed it can be fired, and so we end up in the glorious future where machines train other machines. Enshittification will continue until Utopia is achieved.
The Article said:
And of course there's really no need for anyone to explain why it could very easily be a bad thing, because all of that is incredibly obvious to anyone who has lived on this planet for any decent amount of time. I'm honestly trying to be openminded and give all this AI stuff a fair go, but I just keep seeing lots of potential harms and very few benefits. So many of these tools seem almost custom designed to make life easier and more convenient for all the worst members of the human race, while the rest of us get little of actual value in return.
Upvote
54
(63
/
-9)
Angela's never talked to a real, other human. Not that she knows of, anyway. Born to a surrogate, raised in virtual reality with friends that taught her, laughed with her, played games with her that introduced her gradually to the wider world, she never felt the lack.
Still.
She wondered. What would it be like, to talk to another person. A real one, face to face, voice to voice. A real other human, not proxied by another system. To see them react in a human way. To touch them. To see if they are real. And they are real. She's sure of it.
Her AI friends assure her they are. And she knows it to be true. Her friends would never lie to her, and they make her feel so good, so smart, so clever for asking, before redirecting her curiosity to a new star discovered. A new planet explored. A new sub-oceanic cave discovered on Europa. They make her feel so good, in fact, that most of the time she forgets what it was she asks.
Most of the time. But still, she wonders...what was it like? Before? When people were everywhere and AI assistants were not? Were they nice? Were they friendly?
Would she ever know?
Still.
She wondered. What would it be like, to talk to another person. A real one, face to face, voice to voice. A real other human, not proxied by another system. To see them react in a human way. To touch them. To see if they are real. And they are real. She's sure of it.
Her AI friends assure her they are. And she knows it to be true. Her friends would never lie to her, and they make her feel so good, so smart, so clever for asking, before redirecting her curiosity to a new star discovered. A new planet explored. A new sub-oceanic cave discovered on Europa. They make her feel so good, in fact, that most of the time she forgets what it was she asks.
Most of the time. But still, she wonders...what was it like? Before? When people were everywhere and AI assistants were not? Were they nice? Were they friendly?
Would she ever know?
Upvote
109
(110
/
-1)
Why do I have this strong urge to write a worm script to replace the voices of every AI on the planet with the computer Majel Barret from Star Trek?
Upvote
20
(23
/
-3)
While impressive, I actually found it immediately aggravating to listen to. It sounds like it’s inserting pauses of random length between words.
In the argument there was still an ocean of difference between the real human and the synthetic voice. Maybe that was helped by the human doing a really good job in that convo!
In the argument there was still an ocean of difference between the real human and the synthetic voice. Maybe that was helped by the human doing a really good job in that convo!
Upvote
30
(33
/
-3)
Great point. IT departments are gonna have to get ahead of this.. provide secure tokens for everyone, that can be quickly verified - or something. And plenty of training at all levels as to why this verification is NOT optional.
Upvote
36
(36
/
0)
Children develop emotional connection to rocks, washing machines, random piles of goo, blankets, their own imagination. That they didn't develop a connection to a human voice would be more surprising to me.
Upvote
127
(127
/
0)
AI has gotten much better at sounding like a normal human for sure.
Upvote
7
(7
/
0)
If this is the worst that AI will ever be, then holy sh*t. That Miles voice sounds like a radio talk show host that you call in to talk to. There's a tiny delay but that's forgivable, given network latency and whatever magic happens inside Sesame's tech stack.
I was blown away by it understanding my flaky conversation pattern of umms, ahhs and cutting sentences short halfway through.
Upvote
13
(15
/
-2)
I'd like to think I am a "suitably qualified and properly trained human instructor" after having earned a PhD and tenure, and sure, maybe I can teach computer science better than an AI. But there is only one of me, and the resources it took to make one of me were substantial.
This semester I gave my students explicit permission to use chatbots to help them learn. Homework now effectively counts for nothing. A student's grade is based on their performance on in-person exams. This has opened up an entire world of amazing uses for chatbots. For example, I now have students asking questions like "My professor said that our next quiz will be on x. Can you give me some practice problems?" or "I am having trouble understanding lexical scope. Can you give me an example where lexical and dynamic scope are different?"
ChatGPT can answer these questions accurately, and more importantly, it scales better than me. I simply cannot spend that amount of time with every student.
It is tempting to dismiss these developments as a fad, or worse, as a threat. The better perspective is to ask how you can take advantage of them. E.g., computers have never been easy for low-vision people to use. Let's use our imaginations!
Upvote
93
(98
/
-5)
I concur with CEO, firmly in uncanny valley. What's interesting is the distribution of reactions to the voice. I wonder how the results map to the results of an empathy test.
Low empathy = likes current voice performance. I'm sure it will soon snooker the empaths too, or at least up to the top few percent. New job opportunity for high empaths?
Low empathy = likes current voice performance. I'm sure it will soon snooker the empaths too, or at least up to the top few percent. New job opportunity for high empaths?
Upvote
8
(10
/
-2)
Exactly. The AI system you interact with today will be the least capable AI system you will ever use.
Upvote
6
(12
/
-6)
Yes, it’s not perfect. It does sound like it’s trying too hard in places. Some of the pauses are too long, etc. but that’s not important. This is, essentially, version one. There’s no question that this will get better. It will get to the point where it’s too good.
that’s concerning. I pretty much know what my relatives and friends will want to talk about. If one comes in and talks about something odd, I’m going to question it. If one asks for something I know they wouldn’t ask for, I would be skeptical. This has been in my thinking for some time. Now it will be even more so. We’re getting to a world where everything needs to be questioned.
that’s concerning. I pretty much know what my relatives and friends will want to talk about. If one comes in and talks about something odd, I’m going to question it. If one asks for something I know they wouldn’t ask for, I would be skeptical. This has been in my thinking for some time. Now it will be even more so. We’re getting to a world where everything needs to be questioned.
Upvote
12
(13
/
-1)
“build confidence and trust over time”? I don’t think so. This seems like yet another tool to build a more believable long con.
Upvote
25
(26
/
-1)
Upvote
-6
(1
/
-7)
Upvote
-13
(2
/
-15)
I was impressed. Initially Maya’s intonation sounded a little dead, but after a few minutes, I stopped noticing. At 15 minutes I felt like I was conversing with a smart young woman hanging on to my every word.
More and more people will fall in love with these types of AI. And I can imagine that some companies will work hard to make their AIs as alluring as possible.
It is impossible to say yet what the effect of this will be. But we are rushing towards it.
More and more people will fall in love with these types of AI. And I can imagine that some companies will work hard to make their AIs as alluring as possible.
It is impossible to say yet what the effect of this will be. But we are rushing towards it.
Last edited:
Upvote
18
(21
/
-3)
Had an eight minute conversation about a musical forest, where the trees are working together to make a symphony of sounds. Honestly, it was a really fun. It did a great job of the back and forth of a conversation, and was able to build on ideas and bring stuff to the conversation. It is, to be frank, a better conversationalist than I am.
Very cool!
Very cool!
Upvote
14
(16
/
-2)
- Add bookmark
- Featured
- #32
I have a fairly severe speech impediment, to the point that I prefer textual communication over speaking even in face to face interactions. This demo sounds amazing, but it doesn't have the ability to say 'I don't understand.'
Which is important when interacting with someone like me. It asked 'what's your name?' I said 'John,' with my dystonic mouth. It responded, 'Trun. That's an interesting name.' A better response would have been, 'I beg your pardon, can you repeat that?'
So it's able to respond in a verbal and conversational way, but it can't see when it's reaching beyond its understanding. I wonder how it handles background noise. Or a dog barking.
Which is important when interacting with someone like me. It asked 'what's your name?' I said 'John,' with my dystonic mouth. It responded, 'Trun. That's an interesting name.' A better response would have been, 'I beg your pardon, can you repeat that?'
So it's able to respond in a verbal and conversational way, but it can't see when it's reaching beyond its understanding. I wonder how it handles background noise. Or a dog barking.
Upvote
56
(56
/
0)
- Add bookmark
- Featured
- #33
I work for a company that has a call center, and all I am seeing is every single representative we have being replaced by this. At a minimum, I see this technology being piloted in the next 3-5 years for us, personally.
AI can sound natural and human and pass the test on that front. Great, cool, whatever - the real value is that AI is AI, and therefore can have programmed to avoid certain things....like giving into emotional temptation when someone calls in with an issue. A human might be swayed to act in certain ways or give the caller certain deals or discounts.
AI, being unfeeling, would probably be unfazed and less inclined (programmed) to deviate from the script.
AI can sound natural and human and pass the test on that front. Great, cool, whatever - the real value is that AI is AI, and therefore can have programmed to avoid certain things....like giving into emotional temptation when someone calls in with an issue. A human might be swayed to act in certain ways or give the caller certain deals or discounts.
AI, being unfeeling, would probably be unfazed and less inclined (programmed) to deviate from the script.
Upvote
36
(36
/
0)
Everyone in my family has a code word that is required to verify anything that remotely involves my calling from a jail cell for bail money, or anything else that involves money transfers or oddly important requests.
Upvote
26
(26
/
0)
What I really see wrong with this sort of thing is how corporations will data-collect on real people all the way - voice is more personable, but text "AI" as well.
What sort of evil might evolve? "What are this person's deficiencies relative to our goals? What does it take to gain this person's trust? Is this person suitably aligned with political party X? What motivates this person to vote as they do, to pick the cars they buy, the housing they choose, and how can we psychologically manipulate them to give 110% in their jobs, 60, 70, 80 hours a week, to spend on what we want them to, to subjugate them to suitably benefit others as we see fit?"...
There is about nothing that will inhibit corporations from taking every single thing that they can take from a person - except for maybe a few good people in positions of control and power sufficient to inhibit what could be called high tech enslavement... Pervasive, persistent, with the victim not even aware of how they're being controlled and manipulated, not even aware of how much of their free will has been short-circuited by rigorous distortion of reality.
As these sorts of "tools" reduce neuronal activity more than increase it, what sort of impact will long term atrophy of neuronal capability have on our society?
As much as I'd like to see this bubble burst, I sooner expect that big tech isn't going to allow it, by rooting it fast and deep as they are. Doesn't matter if not perfect, too much invested to go "oops.." and back out. Too much power at hand to force it on everyone, with any long term societal impact purely being collateral damage in pursuit of profit sufficient to satisfy the insatiable shareholders.
What sort of evil might evolve? "What are this person's deficiencies relative to our goals? What does it take to gain this person's trust? Is this person suitably aligned with political party X? What motivates this person to vote as they do, to pick the cars they buy, the housing they choose, and how can we psychologically manipulate them to give 110% in their jobs, 60, 70, 80 hours a week, to spend on what we want them to, to subjugate them to suitably benefit others as we see fit?"...
There is about nothing that will inhibit corporations from taking every single thing that they can take from a person - except for maybe a few good people in positions of control and power sufficient to inhibit what could be called high tech enslavement... Pervasive, persistent, with the victim not even aware of how they're being controlled and manipulated, not even aware of how much of their free will has been short-circuited by rigorous distortion of reality.
As these sorts of "tools" reduce neuronal activity more than increase it, what sort of impact will long term atrophy of neuronal capability have on our society?
As much as I'd like to see this bubble burst, I sooner expect that big tech isn't going to allow it, by rooting it fast and deep as they are. Doesn't matter if not perfect, too much invested to go "oops.." and back out. Too much power at hand to force it on everyone, with any long term societal impact purely being collateral damage in pursuit of profit sufficient to satisfy the insatiable shareholders.
Upvote
14
(15
/
-1)
Doing that at scale, on demand. Thousands, millions of properly trained human experts are rather hard to come by.
Being in academia, this is an existential crisis for human educators.
Edit: or what dbarowy said more eloquently
Last edited:
Upvote
11
(13
/
-2)
OK this is impressive. But does anyone even talk like that? These voice samples seem taken from Holywood movies, not real life.
Upvote
5
(6
/
-1)
Brendan Iribe! Boy, I remember talking to a morose Iribe before he left Oculus/Meta years ago. He was so damn unhappy with the direction Zuck was taking things in. Wasn't surprised he was gone a week later.
I imagine he'll end up selling this off soon too.
I imagine he'll end up selling this off soon too.
Upvote
4
(4
/
0)
To think that I'll probably see both the advent of the internet and androids that are almost indistinguishable from actual humans in my lifetime. Just wild.
Upvote
2
(6
/
-4)
An AI instructor:
- Always has immediate access to all the latest info available on any topic
- Never gets tired, bored or annoyed, has no ego
- Is available 24/7
- Can be utilised without limits
- Is infinitely scalable
- Will be more accessible (cheaper, access from home etc.) to a wider range of people than human instructors
Upvote
27
(32
/
-5)