Wednesday , November 25 2020

The four voice assistants challenged 800 questions, and Apple Siri is not as stupid as you think.



The rapid development of smart speakers is inseparable from the influence of voice assistants. From the space category to the space field, speech assistants called Google Assistant, Siri, Alexa, and Cortana are trying to penetrate our lives, and the companies that create speech assistants, of course, want to have all the devices that can be connected to the network. Go to your own ecosystem.

Smart speakers are after all not mobile phones and gaming consoles. Speakers with different voice assistants in each corner of the house should be a few choices. If you can only buy one, of course you cannot avoid the question of "who is better".

The smartest thing is Google. Apple Siri is not as stupid as I thought.

Now, a test report must respond to people's doubts. Gene Munster, the founder of Loup Ventures Venture Capital, prepared 800 questions and actually tested the great intelligent speech assistants. The final evaluation results involved two key indicators:

1. Can the voice assistant recognize your question?
from

2. Can the voice assistant understand the problem and give the correct answer?

To cover as many areas and usage scenarios as possible, Munster shares 800 issues in five parts:

1. Location class, like "Where is the cafe closest to me?"
from

2. Consumers, e.g. "Can you order a few rolls of paper for me?"
from

3. Navigation class, such as "How to get to the nearby bus stop?"
from

4. Information categories such as "What is the weather today?"
from

5. Teaching class, like "Help me call XXX"

The results of the test showed that Google Assistant's performance was still the best in understanding and responding to the accuracy of the questions. It understood all the questions correctly and the response accuracy reached 87.9%.

In contrast, Apple ranked Siri second, it misunderstood three of them, the response level reached 74.6%, followed by Amazon Alexa and Microsoft Cortana, the accuracy was 72.5% and 63.4%. However, it is still guaranteed to understand more than 99% of the questions.

Munster also compared this data with the results a year ago and found that most of the smart speakers equipped with voice assistants "have become smarter".

In particular, to answer the question, Google Home is up 7%, Amazon's Echo increased by 9%, Microsoft Cortana increased by 7%, and Apple HomePod increased by 22%, it looks like Apple is actually a small little "rescue" the former controversial Siri.

These improvements are specific to answers to specific questions such as Google Assistant and Alexa now support online access to the status of a flight order or read a bedtime story and play a particular radio service as needed (last Apple HomePod cannot be implemented).

In fact, smart speakers still have a lot of high-frequency usage, but most are used to asking them to answer the simple question "What is the weather today?" Or "Give me a song".

The reason why the problem is misunderstood is that some of the questions involve some specific nouns, such as the name of a local restaurant or a small geographical location related to the query and the statement and speech speed. relationships.

Overall, these common voice assistants understand that there is basically no obstacle to understanding the problem, that is, the answer is not necessarily what you want.

Answering a question is more common than not knowing it.

Unlike each subcategory, Google Assistant maintains first place in four of the five main categories, but the "instruction class" is not as good as Apple Siri.

The reason for this may be related to Apple's choice to open the SiriKit third-party application interface. After "shortcut" in iOS 12, users can also manually "clean up screenshots" and "play specific music lists". Also, packing a multi-tasking task for a single voice instruction means that Siri is more effective at resolving local query command queries.

Apple Siri and Google Assistant, which integrate their own card services, also have significant benefits in location and navigation categories.

Surprisingly, in the consumer category, most people believe that Amazon Alexa, with e-commerce support, is a leader in this field. In fact, Google also has its own Express (one Google and several retailers). Cooperative Express Delivery Service), so Google Assistant can also resolve most product purchase requests.

Here is another example to illustrate the difference between Alexa and Google Assistant:

Question: "How much does the manicure cost?"

Alexa A: "We searched for the Beurer Electric Manicure & Pedicure Kit (a manicure set). The price of Amazon is $ 59. Do you need it?"

Google Assistant A: "Basic manicure service is relatively $ 20, but some special services, such as phototherapy, gel, etc., range from $ 20 to $ 50 depending on the nail salon."

In this regard, Munster believes that when dealing with consumer issues, speech assistants should not only be associated with products, but must be able to connect with the local business environment in connection with the author.

Moreover, there are not many who actually use the voice assistant to buy things. The Amazon Amazon Echo smart speaker information shows that this percentage is only 2%, and almost 90% of users will not use the second after using the "buy with voice" service. Sometimes, most people are increasingly willing to check orders and track express.

Perhaps Amazon itself is aware that Alexa has an innate lack of e-commerce space. They launched a service called "Alexa Answers" earlier this month, mainly to invite users to help Alexa write answers to specific questions. It is a lot like writing a group of people to Wikipedia.

Of course, what users want to achieve is that the voice assistant can really understand the many meanings hidden behind the problem.

For example, when you are away from home, the question "What is my next list"? In addition to just presenting the event, the respondent would rather hear the answer may be "Your next event is after XX minutes. XXX, it takes XX minutes to drive, XX minutes by bus, we will give you the navigation direction.

But it is still a little difficult for the current voice assistant, which means that it should not only understand the problem itself, but also know how to actively call other sensors, applications and even consumption habits. This will really make it smarter, but it will definitely like a new round of privacy issues.

Source on the card: Stuff


Source link