Huge potential Cortana feature idea - Listen in during calls and texts and suggest Reminders / Appts

[tacroy]

So I had a crazy idea this morning that I really think could set WP apart.

Let us be able to turn on an option in Cortana to allow her to listen into what you say on your phone calls and texts and create a list of reminders / appointments / etc for you.

So for example, I'm talking to a client on the phone and say "Sure, I will call you next tuesday around 6 about that meeting." and then later say "Ok, great, I going to upload that document later today". Once I get off the call Cortana will pop up with a list saying "I noticed you mentioned a few things, would you like me to remind you?" and then it would have a scrollable list that in this case would include "Create an appointment to call Client Tom tuesday at 6pm" and then "Reminder to upload document for Client Tom. What time should I set that for?"

User voice link to get it in front of MS.

Give the option for Cortana to listen into phone calls and texts and make Appointment / Reminder recommendations when call has finished. – Cortana Feedback

 

[anon(8657436)]

This is highly ambitious but yeah awesome as hell!!! This looks right out of the future. I don't think there was ever such excitement for any other Digital Assistant.

 

[tacroy]

I really feel like this is something reasonable given the current backend in place. Essentially, when I press the "Call" button, the background process for cortana opens and starts parsing what I'm saying exactly like how it does now, just in the background and as a list of tasks instead of as a single task. Then when I press "Call end", present a scrollable list of tasks.
I wonder how open the cortana API is, I might just make this myself...

The largest programing challenge would likely be how to find the "breaks" between comments to allow it to parse throughts. Since right I believe they use a second or two of silence and then parse the entire audio file.

 

[tacroy]

Also, it would only use the Microphone input so it only listens to my audio so there is not a privacy concern.

 

[Alex Blast]

This also means if this feature is to be implemented, the likelihood of non-English speaking people will be experiencing it anytime soon is rather little, no?

Sent from my RM-846_apac_vietnam_203 using Tapatalk

 

[omgitsnick]

Actually it already scans SMS, like it puts a dotted underline on time related subjects. Like if you text "Lunch at 1?" and tap on "at 1" it will create an event on that time either on the same day if it hasn't passed 1 o'clock, or it will create it the next day. It creates events/appointments but not reminders.

 

[slick5150]

This would be a big data and battery hog. It'd have to constantly be sending data back to the server to turn your speech into text Cortana understands. So if you're on a 30 minute call, it'd be sending data for 30 minutes as well.

 

[gwheatle]

I see no harm big brother does it already

 

[Noahma]

The only problem I see with something like this is the legality. In the U.S. there are laws that are individual by state that would make this function a legal nightmare. I live in Colorado where I believe both parties are required to give consent for a call to be recorded. That right there would make this a HUGE headache for Microsoft. If the mic was set to only listen to MY voice I think that would work.

 

[Deofol]

This would be a big data and battery hog. It'd have to constantly be sending data back to the server to turn your speech into text Cortana understands. So if you're on a 30 minute call, it'd be sending data for 30 minutes as well.

Well yes and no. Yes if it remains software based (which it wont).

What we are going to see in the next coming years is a dedicated core for AI (and other local processing).

What is done now by software will evolve into a AI acceleration chip. Just like the evolution of 3D GPUs. Remember the Voodoo cards of old?

Hardware based AI chips will supersede software systems like Cortana. They will have the ability to listen all the time, less stutter, better efficiency, and an accuracy level unseen by the current generation; less dependent on the cloud. Ie. True autonomy.

So the OP is right on.

If you want to push this all a bit further (and I know you do), the concept now of packing as much processing power into a device (albeit Surface Pro 3 is awesome, and at this time the most advanced computer system on the planet) will become irrelevant. Current LTE rates top 16Mbs and with XLTE you are approaching 32Mbps. At this rate, and the speed of light (670,616,629 MPH) you are more than capable of creating a true cloud based experience.

Let me explain further. Its all going to come full circle and our devices will simply be dumb terminals with specialized chips doing the logical local grunt work. Latency will be a nonissue. It takes light from the moon about 1.2 seconds, this is the same rate of speed your desktop (in the cloud) will travel to you. Without going into boring details, your cloud account will "hop" close to you as you travel in order to reduce latency.

So essentially, Windows (amongst others) will become (for lack of a better pun) a window into a super computer in the cloud that you don't have to manage. This concept works for much of the computing space, but physical computers will still be required for things like live audio/visual production, development, and where latency must exist under 10ms, but it is a specialized space that most people will not need.

So why will this enhance the computing experience?

Well, basically it means that every device you use will be just as powerful as your cloud hosted device. It will be true singularity in your experience. Battery life will sore, since we no longer need the space required for cooling a super high speed local CPU, you will have much more room for battery space and other "helper" chips that do things like local voice recognition and other remedial tasks.

You should see all of this in the next five to ten years.