Video: Cortana vs Google Now

[RumoredNow]

The players:

• Cortana by Microsoft on a Nokia Lumia 925 (T-Mobile branded) running Windows Phone 8.1 GDR 1 Preview for Developers
• Google Now by Google on a Samsung Galaxy Tab S 10.5 running Android 4.4.2 w/ Touchwiz

The camera: I used the wife's Galaxy Note 2 to film this. Sorry about any lack in quality, I'm not practiced with the device and it doesn't seem to autofocus as well/quickly as my Lumia 925.

The same twenty questions were asked of each Virtual Assistant in turn and the results were scored (by me). The questions were what I felt was a good mix and the sorts of common uses people would ask of their device. For the constraints of this video everything was done on a desktop and there was no actual field use for navigating and geofencing; although those abilities do come up in the demo - I just wasn't driving around town while I filmed. Both devices had been "primed" before hand by filling in the interests and settings.

Here's my BLOG feed if anyone prefers to read a bit more about the history of the two assistants before viewing the video: My Alter Ego

The video itself is about 45 minutes. Hey, that was 20 questions each (40 total) plus responses by the devices, my reactions and scoring the results... I feel this is a fairly thorough test, even if it is off the cuff, and better than a quick 5 - 10 minute shoot.

Hope you enjoy this presentation.... And let the comment war begin. :winktongue:

 

[Silviu Bogusevschi]

Pretty interesting comparison. I did watch the first 10 minutes and scrolled to the end to check the score.

So, Cortana FTW

 

[RumoredNow]

Pretty interesting comparison. I did watch the first 10 minutes and scrolled to the end to check the score.

So, Cortana FTW

LOL spoiler alert...

 

[tgp]

I did a quick little test of my own just last evening. Cortana was on an AT&T Lumia 920 with the latest DP update, WP8.1 GDR1. Google Now was running on my Nexus 5 with KitKat 4.4.4. Both devices were on my home WiFi.

I don't have any scientific results, but accuracy was roughly equal. Out of about 10 questions, Google Now missed one and Cortana missed one.

Voice: both quite good, but Google Now is more natural sounding. It has better inflections as well. Cortana sounds a bit more computerish. Not by much though; it's pretty good.

Speed: Google Now responded quicker, sometimes significantly quicker.

One thing that was interesting was when I asked if I need to use an umbrella tomorrow. Cortana said I don't need one, but Google Now said I would need one. And they both showed exactly the same weather prediction! However, even though they showed the same weather, I'd give the edge on the need for an umbrella to Google Now since the chance of rain was 40% I believe. But, that one could go either way. I suppose they just have different thresholds for recommending an umbrella.

All in all, I'd call it a tie. Cortana is indeed impressive for the short time it's been around.

 

[RumoredNow]

Could you have set that up to be any MORE in favor of your phone?

I tried to be as honest as I could about it. Neither I, nor my wife, have built up any usage of Google Now previously. Although with Google's obsessive tracking, my wife's long standing Google Account should have helped Google identify her interests and I've pushed her to use Google Now for the 3 days she had owned the tablet at that point. I also balanced that by a hard reset of my Lumia to clean it out from the last 4 months usage. Additionally, Google Now has had 2 years to advance itself vs Cortana's 4 months...

Geez. Some points-
The video was WAY too long. 42 minutes! Should've been 5-10 minutes.

There are already a slew of quickie videos on the subject. Why should I repeat that formula? I warn people ahead of time how long it will take and there is the option to fast forward on the nav bar of the video... I wanted something more in-depth than a sneak peek sort of thing.

That tablet was brand new and never left your house. Is Cortana's contextual learning ONLY on your device and 100% offline?

I'm not sure what you are asking there...

If you ship me a device that has a more "used" Google Now on it. I'll reshoot... Or you could try and shoot a video of your own in counterpoint.

To what does the 100% offline refer to?

You totally downplayed always-listening hands free technology? Seriously??

If you would have watched the video, you would see that the "always listening" feature could actually be a detriment. There were several times when it kicked in even though the words "OK." "Now" and "Google" were widely separated within a sentence. It actually listens too hard and triggers during casual conversation.

That was just the first 60 seconds I couldn't watch the rest, it's too long. I'll stick with the dozens of other vids that have been done already.

So, after 1/45th of the video (your estimate, not mine) you show your unwillingness to keep an open mind on the subject and a complete lack of patience to view something on a subject you seem to be passionate about...

Um, OK.

​Thanks for trying though.

 

[RumoredNow]

Cortana scores a win because it it gave you travel directions after you asked about kayak?? Are you joking? I'm pretty sure you can't fault Google's coders for not anticipating ridiculous things nobody normal would ask.

No... Google lost because it interpreted "How long will it take to kayak from Istanbul to Athens?" as a request to book a flight on the kayak travel site. Not the same thing at all. Cortana won as the first answer correctly interpreted "how long" as the interrogatory and therefor the main thrust of the inquiry. Neither answer was correct, but Cortana more accurately understood my language and put the words into context. Google's ad driven mentality assumed I wanted a product that pays them advertising revenue.

A portion of the questioning was geared toward odd requests... Because that is what real people do with their devices in the real world. They act as themselves rather than encoding their behavior to please a machine that is supposed to serve them, not vice versa.

Have you never observed that humans can be whimsical and illogical?

 

[smoledman]

Lol OK I pained myself to watch a bit more. Cortana failed at finding your movie times in an epic way and your response was to TOTALLY bash how Google came through. Any time Google Now did some small insignificant thing you didnt like you said "epic fail". Omg... What a biased review.

Exactly. Cortana is maybe 70% of the way there. It's in BETA, I wouldn't expect it to be matching Google Now in functionality until the end of the year.

 

[trekgraham]

Cortana keeps impressing me, Google Now and Siri have a head start. What Microsoft needs to do is work hard and fast on combining APIs of Windows 8.1 and Windows Phone to makes universal apps which will maybe make developers more interested in developing official apps.

 

[RumoredNow]

Cortana keeps impressing me, Google Now and Siri have a head start. What Microsoft needs to do is work hard and fast on combining APIs of Windows 8.1 and Windows Phone to makes universal apps which will maybe make developers more interested in developing official apps.

The wind is blowing in that direction... Seamless integration across every screen is the way of the future. I believe Microsoft understands this. I look forward to Windows Phone converging with Windows proper. LOL watch out.. *wink wink* A MSFT powered watch is on the way already...

​Up Next: MSFT TV???

 

[trekgraham]

Why would anyone trust a company that is in the business to sell your information. Trusting Google is very risky business since they make millions as a global search engine. That would be like having prisoners guard the prison. I don't believe Microsoft is finished at all. Windows OS is still the leading operating system in the PC market. And like it or not but the PC is not dead as there are still things that cannot be done on a tablet or a smartphone.

 

[RumoredNow]

What does WP offer customers that's not already available on established and trusted platforms? The answer is nothing.

Yet again, you fall into the fallacy of assuming your personal tastes are absolutes and no one should be able to see it any other way.

What Microsoft offers me that is not readily available on other platforms (let's leave the established and trusted parts of your statement out as subjective qualifications) is a UX that I enjoy. It works for my needs and fits the way I use my device. That is what people want. Some find it on one platform, some on another.

​'nuff said.

 

[smoledman]

Doubt it. All of Microsoft's divisions got hammered by the competition this year. That's pretty grim. Xbox, Surface, WP all took a beating. Layoffs. I see a future Microsoft being repurposed more the way of IBM and Blackberry with a tighter smaller focus that's still viable but a shade of their former self.

That's the crux of the matter. What does WP offer customers that's not already available on established and trusted platforms? The answer is nothing. I really can't see MS ever being a tech superpower ever again. That ship has sailed, and MS will have to be content repurposed as a smaller IBM or blackberry type of company. The next challenger to Google and Apple is going to end up being something so revolutionary it changes the entire paradigm of how we compute.

​You forget that MS owns 30K patents, and are ready to unleash them.

 

[AG VK]

​You forget that MS owns 30K patents, and are ready to unleash them.

Yes, that's all they're good for now is it? Patent trolling, the last refuge of the incompetent. And you actually support this?

Sometimes the level of fanboyism among these MS/WP supporters is disgusting. (I would say paid shills and astroturfers, but I think Redmond is smarter than that).

 

[AG VK]

Yet again, you fall into the fallacy of assuming your personal tastes are absolutes and no one should be able to see it any other way.

What Microsoft offers me that is not readily available on other platforms (let's leave the established and trusted parts of your statement out as subjective qualifications) is a UX that I enjoy. It works for my needs and fits the way I use my device. That is what people want. Some find it on one platform, some on another.

​'nuff said.

Yes - and clearly that is something that 3% (and declining) of smartphone users want. If MS wants more than a negligible share of the market, they need to do what the other 97% want.

 

[smoledman]

Old news, they've been patent suing android and Apple for years, look where that's got them: PC sales at record lows, more dwindling of mobile market share and PS4 sales trouncing Xbox One.

​PC sales are not at record lows, in fact they've been bouncing back the last 2 quarters. Tablet sales are down, while smartphones remain strong.

 

[TechFreak1]

I had a quite a chuckle at the beginning (as well as around 10 minutes) and it was a decent video in my opinion, thank you for taking the time to record + share. Google does try too hard to listen, the sensitivity is insanely high then again it is most likely set up like that on purpose? :winktongue:

What really shows is the insane amount of work MS has put into Cortana, when she was until recently only officially available in the US as a beta. Whereas Siri and Google now have been around for much longer therefore had much longer to polish + fine tune.

Cortana will improve over time and will surpass Google + Siri pretty quick providing MS keep pushing as well as innovating at the current rate.

 

[Ordeith]

Nice video.
Just a note on the "I need a bus to the mall" question. Cortana probably got that right. First she needs to know which mall, then she gives you transit directions to your selection.

 

[RumoredNow]

Nice video.
Just a note on the "I need a bus to the mall" question. Cortana probably got that right. First she needs to know which mall, then she gives you transit directions to your selection.

​Certainly... And I knew I could get to the Nav I wanted from where Cortana answered me. It just was not as "immediate" as Google Now. *shrug*

 

[Ordeith]

​Certainly... And I knew I could get to the Nav I wanted from where Cortana answered me. It just was not as "immediate" as Google Now. *shrug*

I think it's a matter of work flow. Do more people know the mall they want to travel to than want all possible routes to multiple possible destinations? I would think so.
Also, When you need a bus to a specific place where there is only one possible destination Cortana goes right to the transit directions.

 

[RumoredNow]

You may be right about Cortana...

However, there is but one mall within reasonable bus distance of where I reside and Google Now nailed it. The other options presented by GN were popular anchor stores at that same mall to help decide which stop I most desired. I really thought Google Now did a better job on that one. Given that the question is vaguely worded on purpose as part of the experiment, Google Now made the better interpretation of my perceived needs, IMHO.

Now those over at AC who are contesting points awarded in the video would never reference my defense of Google Now scoring a point over Cortana in this thread. Did you not know I am horribly biased against GN in their eyes?