Messaging apps unreliable due to WP's broken WiFi connectivity design (Audio API not Whatsapp fault)

[ah06]

Ok, first off you just lost me here. I suddenly became so uninterested reading the rest. First you're saying that other OS keeps some sort of method to have an ongoing connection for the apps to use, and WP is the only OS that doesn't. Then you stated that basically in a "no-network" situation, mentioning WP8 in particular, makes all IM apps to render useless.

Now tell me, how can any OS, basically any device that is NOT connected to the network, can send or receive any messages and provide push notifications? Basically all these messaging apps need a data connection to send and receive data, no matter what the OS is.

And if you wanna use mobile data to have real time push notifications, you can set one of your emails "Download new content" to "as items arrive". That way any app can now have access to mobile network in real time. Also I have used Kik before, and it had real time notifications, without using the Music player stream like WhatsApp does. Skype calls and IMs goes through just fine, same with Viber.

Only WP8 by default allows that no-connection situation. Hence mentioned in particular. Other OSes do not allow that no network situation to occur.

Now tell me, how can any OS, basically any device that is NOT connected to the network, can send or receive any messages and provide push notifications? Basically all these messaging apps need a data connection to send and receive data, no matter what the OS is.

Precisely, that is why Whatsapp uses the Audio API. Because all messaging apps need a data connection to send and receive data. All OSes allow this under all conditions, so Whatsapp there doesn't use a Audio API.

WP does not allow it under conditions of no data, screen off (by default). Bulk of Whatsapp userbase falls in this use case. Hence they NEED to use the Audio API to keep a connection alive.

It would have been clear had you bothered to read the first post which you admitted you din't. No point talking to you any further. Do you frequently call people haters without reading their post first?

 

[Daniel Ratcliffe]

What if whatsapp enabled this as an option that is on by default? Would that help you? I would be willing to use it if they would allow me a way to turn off that hack.
Sent from my HTC TITAN X310e using Board Express

 

[ChMar]

You understood it best so far
I am in no way suggesting enabling persistent WiFi would solve any and every IM issue. It would solve this one. Messaging apps are unreliable (cannot be relied upon to deliver messages) for precisely this one single reason. This pretty much entirely explains Whatsapps use of Audio API.

Of course, data is crucial, but I am not suggesting WiFi *Replace* data for IM apps.WiFi is not always available, neither is data. However WP handles data connections well (Does not terminate them). It does not handle WiFi well (it terminates them). As such it can sort of commit total connection suicide. This should not happen.

Whatsapp uses Audio API to prevent this exact scenario.

Let me explain Whatsapp's reasons in even more detail. Whatsapp is the most popular amongst its competitors (Kik, KakaoTalk, WeChat, Line, Hike, Viber etc). However most of these are region specific and often in rich countries or countries with cheap data.

KakaoTalk (South Korea)
Line (Japan)
Hike (India)

Whatsapp is largest and is the biggest BBM drop in replacement. It is used very very widely not only in countries with cheap data but also by millions of people without any data whatsoever. They use it around the house and at work as a BBM replacement. You are right in that there may still be "blackout" periods when they travel or are otherwise out of range but the Audio API prevents any such *needless* occurences when they are IN range. If a sleeping phone terminated connections, 90% of the time you would be out of reach. Now if you're in WiFi range, you're instantly available. This is exactly how million of people are using Whatsapp today.

This might be hard for anyone in a rich country or one with cheap data or free messaging to understand but is the reality of their userbase.

Whatsapp does not use audio to keep Wi-Fi on. Its no longer the case with latest portico update. Whatsapp uses the audio api so it can keep the connection alive. They did not want to change their protocol and use push notifications(like viber and Skype) so they use this hack.

This is how things are for all major mobile OSs. On iphone you say your app is a newsstand and you get to keep data connection alive(this is a conection over TCP/IP socket it does not mean connection through Wi-Fi router or presence of data on your contract). On android you let your background agent keep that connection alive(again this means that the whatsapp is connected throudg their server all the time). On WP you can't do that. To preserve battery WP kills all connections when an app goes to background.

So now you have the following options to make sure you know about your messages:
1. On WP8 you can mask your app as a gps app(again another hack no better than audio api) so now you get to stay in background and poll data faster
2. Use that audio hack stay in the background get to poll data faster
3. Use push notifications just like viber and Skype(no active data connection everything resides on a server fa?ade that only push you notifications not actively pulling data in your phone)

Of course no one knows why they are so suborn and refuse to refactor their protocol to work with method no.3(push notifications). This would benefit all platforms not just wp8(improvements in battery in android and iphone).

But since viber and Skype can do it then I see no good reason why they can't do it and demand that MS gives them something else.

 

[Daniel Ratcliffe]

Play Nice, Be Helpful, and have Respect!​

This means keep the language clean, keep your posts polite, and don’t get into arguments over dumb things. If somebody responds to a thread you posted in poor fashion, ignore it, and the moderation team will quickly clean it up (you can always report a post using the triangle to the top right of the post at any time if you observe anything that is in poor taste/against the rules). Think back to what your parentals and teachers used to lecture - if you don't have something nice to say, don't say anything at all.

Always remember that we are all unique and at different levels here - some of us are experienced Windows Phone users and forum junkies, while others are just starting out. If you’re on the experienced side, have patience for those just joining in on the fun and be helpful. With a little patience and a little help they won’t be newbies for long. Also remember that it is our differences that keeps this place interesting. If everyone was from the same place and at the same experience level it would be boring. This is a worldwide community united by a common interest - our love, or addiction as some would call it, for the Windows Phone!

Seriously though, any more problems and we'll be handing out infractions.

 

[rockstarzzz]

After reading this thread I am not sure why Paladinleeds hasn't closed it yet. Other than some awful posts that are thankfully deleted, this thread serves no purpose. As ChMar and a5cent have precisely pointed out there is no reason to defend whatsapp. It doesn't "need" it, it wants it.

I don't have WiFi at home. How do I use whatsapp? On data connection which is always on 3G or 3G+ and still whatsapp notifications are unreliable!

I don't see how any user would "accidentally" untick that deeply buried option which at least 40% of readers on this site had to dig the first day after the update was released!

As ChMar pointed out - the reason audio "streaming" as been still on going is not just to keep WiFi alive but also to keep the connection alive. All they need to do is invest in servers and allow push like Skype, Viber and make two versions - one for WP7.x and one for WP8.

I understand that if you don't have data plan then you rely totally on WiFi but as a5cent has said earlier - what is the most available connection? WiFi or 3G? Unless you are self employed and get food delivered to your door step, I don't see how you won't leave a WiFi zone at some point. So anyone dependent so much 24x7 on an IM app MUST have data connection no matter what phone he is using and what OS is available to him!

 

[ttsoldier]

What if you don't have Wi Fi?! Then you have to rely on data all the time. I do not have Wi-Fi at home and where I sit at work, I get no Wi-Fi. I rely heavily on Data only and I'm pretty sure I'm not the only user in the world who does this.

The only thing the audio API is good for is killing my battery!
A serious/heavy IM user would NOT rely on Wi-Fi and would have some sort of minimal data package as IMing does NOT require that much data. Hotspots aren't always available.

Your point, and this thread, is moot.

 

[a5cent]

Whatsapp does not use audio to keep Wi-Fi on.
....
Whatsapp uses the audio api so it can keep the connection alive. They did not want to change their protocol and use push notifications(like viber and Skype) so they use this hack.

That it's 'just' about the open IP socket makes a LOT more sense than the strange 'keep WiFi alive' argument. Thanks for the explanation. I suspected as much.

But still... what a disgusting hack!

So, what changed with portico?

 

[ChMar]

That it's 'just' about the open IP socket makes a LOT more sense than the strange 'keep WiFi alive' argument. Thanks for the explanation. I suspected as much.

But still... what a disgusting hack!

So, what changed with portico?

Portico changed that always Wi-Fi on. That is useful for apps that run under lock screen and so they are still valid foreground app that can do whatever they want except drawing on the screen. The always on Wi-Fi solved the issue with reconnecting through your router again and some bad case of coding(more common than one would think).

I make a streaming audio app. I don't account for connection drop so when Wi-Fi goes of I get an exception. Untreated it will crash my app(it will appear in logs so I will correct it later). But what I forget to switch to data connection(simply just reopening my connection) then you won't have audio anymore so my app is pointless I would complain that MS is not doing good stuff.

No there is a more delicate problem and a problem that has a lot of repercussions on push notifications. In order to receive push notifications the phone must connect to MS servers and tell it's ip(be it from carrier if using data connection or ISP if using Wi-Fi). After this whenever a push notifications come the MS servers will ping the phone and alert it that it has data.

When Wi-Fi drops there is a time lapse between the moment the phone reconnects to the MPNS(Microsoft push notifications servers). It needs to do that because your Wi-Fi ip is no longer valid so you won't get push notifications anymore so now it most register the phone for the data-ip(ip assigned by carrier). During this change you go in a state called temporary disconnect. During this stage you receive no more push notifications. After the phone publishes it's ip again to the MPNS then you will received the notifications queued during that temporary disconnect(and the future ones). The queue can handle 30 push notifications.

Now my app that uses push notifications has that server component that tries to push a live tile update. Because Wi-Fi went down and phone is negotiating with the MPNS my server component gets an error when sending the live tile push notifications. If my server component does not see what is happening(check to see if the phone is temporary disconnected or shutdown) my server might stop pushing notifications(it should at least push a toast in case of IM or the MPNS will only queue the tile notifications and not the toast push notification).

Also when the app is opened in the phone it can now fully communicate with the server component and can unblock suspend servicing that were pushing notifications and detected the phone in temporary disconnect state or offline state. Keeping the app and the server component synced is the art of push notifications.

Windows 8 function on somehow similar principle when using push notifications but only queue 1 push (not 30) and has size restriction for payload for push. One can screw with Skype on windows 8 by multitasking(using alt-tab if keyboard is available) and thus disrupting the sync between app and the server component(seams that Skype first pulls all messages and then does the sync). This desync will cause you to receive again some older notifications.

I found this about Skype on windows 8 because I use alt-tab a lot. On wp8 I tend to switch between apps less faster and so I could not observe this desync.

 

[a5cent]

^ Nice bug discovery. Sounds like the protocol used to sync push messages isn't robust enough.

So, are you sure that WhatsApp no longer uses the audio API since portico? You've verified this yourself? I ask because it seems strange. After all, portico doesn't guarantee different behavior. If the WiFi always-on option isn't set, which it usually won't be, then there is no difference to pre-portico.. but then why undo the audio API hack?

 

[ttsoldier]

^ Nice bug discovery. Sounds like the protocol used to sync push messages isn't robust enough.

So, are you sure that WhatsApp no longer uses the audio API since portico? You've verified this yourself? I ask because it seems strange. After all, portico doesn't guarantee different behavior. If the WiFi always-on option isn't set, which it usually won't be, then there is no difference to pre-portico.. but then why undo the audio API hack?

What?

Yes. WhatsApp still uses the audio API

Undo the Audio API because it eats my battery for breakfast, lunch and dinner, spits it back at me and eats it again!

 

[ChMar]

Well Whatsapp still uses it. I doubt they would change that once they opened their mouth and said that it can be done without it. They don't need it in wp7 or wp8.

But as the platform is young and they are top dog they can afford and be insolents and demand api or they won't make changes.

 

[ah06]

Whatsapp does not use audio to keep Wi-Fi on. Its no longer the case with latest portico update. Whatsapp uses the audio api so it can keep the connection alive. They did not want to change their protocol and use push notifications(like viber and Skype) so they use this hack.

This is how things are for all major mobile OSs. On iphone you say your app is a newsstand and you get to keep data connection alive(this is a conection over TCP/IP socket it does not mean connection through Wi-Fi router or presence of data on your contract). On android you let your background agent keep that connection alive(again this means that the whatsapp is connected throudg their server all the time). On WP you can't do that. To preserve battery WP kills all connections when an app goes to background.

So now you have the following options to make sure you know about your messages:
1. On WP8 you can mask your app as a gps app(again another hack no better than audio api) so now you get to stay in background and poll data faster
2. Use that audio hack stay in the background get to poll data faster
3. Use push notifications just like viber and Skype(no active data connection everything resides on a server fa?ade that only push you notifications not actively pulling data in your phone)

Of course no one knows why they are so suborn and refuse to refactor their protocol to work with method no.3(push notifications). This would benefit all platforms not just wp8(improvements in battery in android and iphone).

But since viber and Skype can do it then I see no good reason why they can't do it and demand that MS gives them something else.

Hi ChMar,

Appreciate the detailed explanation. I was under that impression for the longest time before I conducted my own testing and one particular case seemed to me to suggest that they may be able to push notifications even without the Audio API.

If you restart the phone with the WiFi alive option set to "off" and then restart the phone, you will notice that the Audio API is no longer active. The audio API starts it mojo once Whatsapp has been launched once. However, even without the b/g audio streaming, Whatsapp *server* is still able to send messages to the Whatsapp app (real time push messages) either over Data (if its active) or over WiFi (which presumably would be active because screen is on). I kept pushing messages to it for the better part of an hour and it received all of them.

Am not sure what I should conclude from that. They seem to be able to push messages without the API. Are they then using the API just to reduce/optimise/skimp out on their own infrastructure despite having the ability to use push (like other IMs) anyway when the need arises?

Thanks

What?

Yes. WhatsApp still uses the audio API

Undo the Audio API because it eats my battery for breakfast, lunch and dinner, spits it back at me and eats it again!

I know I could spark another massive debate here but according to my testing. Its not the API that drains the battery. Its something else in the app itself.

For example, lets call a phone without Whatsapp installed a baseline case. If you install whatsapp and it runs its streaming in the background constantly (BUT, you do not actually use the app much), it doesn't seem to hit the battery life much relative to baseline.

In my experience, what really is killing the battery life is when you have the app open and are actively chatting/using it. On days when I use it heavily, the battery can go to zero even in half a day (I'm talking constant texting here, almost non stop). Also, when you are IN the app, if you hit the volume keys, you will notice they don't use the background streaming hack (because the app is open and connected and the connection is alive, they don't need to). Yet this 'Whatsapp open and being used' case is the one that is really draining the battery (when the background streaming in not in effect).

 

[a5cent]

So, are you sure that WhatsApp no longer uses the audio API since portico? You've verified this yourself?

Well Whatsapp still uses it. I doubt they would change that once they opened their mouth and said that it can be done without it. They don't need it in wp7 or wp8

Okay, now this is getting confusing:

The issue at hand is WhatsApp's misuse of the background audio API. Apparently, the issue involves portico, otherwise we wouldn't be discussing it. However, the release of portico hasn't affected WhatsApp in any way. WhatsApp still misuses that audio API just as it did before portico was released. So, why exactly is portico relevant to this discussion?

Are you just trying to say that WhatsApp should be using portico's keep WiFi alive feature instead of the audio API, but doesn't?

What?

If you are more precise about what you don't understand, and spare me having to guess, I might actually respond :wink:

 

[ah06]

Okay, now this is getting confusing:

The issue at hand is WhatsApp's misuse of the background audio API. Apparently, the issue involves portico, otherwise we wouldn't be discussing it. However, the release of portico hasn't affected WhatsApp in any way. WhatsApp still misuses that audio API just as it did before portico was released. So, why exactly is portico relevant to this discussion?

Are you just trying to say that WhatsApp should be using portico's keep WiFi alive feature instead of the audio API, but doesn't?



If you are more precise about what you don't understand, and spare me having to guess, I might actually respond :wink:

The consensus seems to be that Whatsapp used the API to keep WiFi alive before Portico came around and gave the option to keep WiFi alive on an OS level. ChMar is suggesting that Whatsapp persists with the API now for a different reasons (apart from sheer laziness) which is to keep the TCP/IP connection alive so that they don't have to use push (and save infrastructure investment on their end).

I think he is suggesting that they used the API for different reasons pre and post portico (To keep WiFi alive before and to keep connection alive now).

However, it seems to me that Whatsapp *server* is able to "push" notifications (i.e Whatsapp app can receive notifications via push) just fine even without the API (which is the case when you just restart your phone and haven't yet opened Whatsapp).

 

[a5cent]

The audio API starts it mojo once Whatsapp has been launched once. However, even without the b/g audio streaming, Whatsapp is still able to send and receive messages

That makes no sense. How can you send a WhatsApp IM without having launched WhatsApp at least once?

However, even without the b/g audio streaming, Whatsapp is still able to send and receive messages (real time push messages) either over Data (if its active) or over WiFi (which presumably would be active because screen is on). I kept pushing messages to it for the better part of an hour and it received all of them. Am not sure what I should conclude from that. They seem to be able to push messages without the API.

It seems one or more of us isn't understanding each other. This is how you've described your test case:

1) You've got a data connection through an active cellular network, which is never disconnected during the test
2) You've got a data connection through an active WiFi hotspot, which is never disconnected during the test (screen is always on)

This is the best possible scenario. You have two possible channels through which data can be transported, both of which are completely reliable. Why would you expect to not receive a WhatsApp IM under such circumstances?

The background audio task is completely irrelevant, up until we encounter a scenario where it is able to prevent the last available data connection from closing. IMHO we already discussed this, so I'm not sure what is getting lost in translation here.

Are they then using the API just to reduce/optimise/skimp out on their own infrastructure despite having the ability to use push (like other IMs) anyway when the need arises?

ChMar suggested it was because the WhatsApp developers didn't want to change the method by which they exchange IMs between devices and their servers. They wanted to "reuse" the methods already employed on Android and iOS, despite WP having its own infrastructure and process for exchanging push messages.

 

[jhguth]

If a developer is concerned about wifi always on not being the default on WP8, it would be incredibly easy for them to add a page during setup instructing the user to turn on always-on wifi for best performance.

 

[ChMar]

@ah06: For an app to be able to stay in the background using the audio api it must first be launched. Push notifications don't require the app to be launched(except once so it can register to the MPNS). On iphone they can stay in background once launched unless user kills it(same on android). On wp8 even with audio bg hack their app might get killed since I can start listening to music and thus kill their app. So they need a fallback. So I guess that their use of push notifications when the app is not in bg is their fallback strategy.

Let's use Skype, mail and whatsapp as comparison. You do not have an active TCP/IP connection for email. You receive push notifications and then open the app and the app downloads the mails. In the UI you see that the mail is not there yet it is downloading and will appear later. But in a sense you are always online when using mail. Let's look at Skype. It's based on a similar principle even though it is a IM app. You receive the notifications that someone wrote to you. One line of text 10 lines it does not matter. You press that toast or open the app and the lines of texts are not there. They are downloading and will get there once the download finishes. You close Skype but you still appear online. Just like in the email case. Because you use push and don't rely on an open active TCP/IP connection(let's call it channel) you can't know if you are online or not and it's simpler to assume that you are online. Mail not being used as an IM it does not have an online/offline indicator. IM do need this. So this is why you always appear online on Skype even after closing the app because it acts just like email. You need to switch account and thus signaling the server that you are going offline.

In whatsapp case thing function a little different. By keeping that channel open(by streaming music) they know when you are online when you go offline and can grab data faster because the channel is already open and data goes through it.

On iphone (at least ios 4.0) whatsapp does not receive push notifications. You have to open the app and not close it so you can receive messages. This is because even there the whatsapp keeps the channel open and communicates through it so unless you open the app there is no channel so no notifications no messages.

So even if whatsapp also uses push notifications they use that channel to speed up message transfer and to monitor the online / offline status of the user.

As for battery drain you get almost the same drain you will get while listening to Pandora for example(a little less since you don't push audio through speakers). Wi-Fi and especially the phone Wi-Fi chips are optimized to go in a dormant state with minimized power consumption(maintaining a connection to the router but not transferring any data) in case the user does not have an app opened that makes downloads/ uploads. Whatsapp does not allows for this since they keep that connection open.

Usually on a "clean wp8 phone" only the OS sends some pings to the MPNS to say that the phone is still active(I don't know the exact time but I assume it's a least 15 minutes apart).

But whatsapp has a strange architecture. Even on iphone the mighty first platform for apps it is faulty(you need to let it run to receive messages). We don't need to talk about android since there the resource management done by OS is very liberal.

But when you have 2 platforms were you keep the channel open you won't change your app just for windows phone platform. You won't care given it's market share. So you will ask for more "open API" from MS. I just hope that in 2 years such apps could be banned from store no matter their name.

No app has any reason to be concerned about the Wi-Fi always on since they cant communicate in the background unless they are music apps or gps apps in which case they will force the connection to remain so no concern there.

The other cases (the apps that stay in BG and run once 30 minutes) are limited in the amount of data they can transfer(depending if they are lockscreen apps or not) so no large data transfer can occur without user consent(meaning also keeping the app in foreground). For more advanced scenarios with no penalty for data-contracts there are other cases.

 

[ah06]

That makes no sense. How can you send a WhatsApp IM without having launched WhatsApp at least once?

You're right. I phrased it very poorly. I've edited them to clarify. What I meant is the Whatsapp *server* is able to push messages to the Whatsapp *client* on your phone even when the b/g Audio API is not being utilised (Which is only the case when you have rebooted phone and have not launched Whatsapp yet)

It seems one or more of us isn't understanding each other. This is how you've described your test case:

1) You've got a data connection through an active cellular network, which is never disconnected during the test
2) You've got a data connection through an active WiFi hotspot, which is never disconnected during the test (screen is always on)

This is the best possible scenario. You have two possible channels through which data can be transported, both of which are completely reliable. Why would you expect to not receive a WhatsApp IM under such circumstances?

The background audio task is completely irrelevant, up until we encounter a scenario where it is able to prevent the last available data connection from closing. IMHO we already discussed this, so I'm not sure what is getting lost in translation here.

Ah coz we're discussing a different thing now. We're now trying to ascertain whether Whatsapp is using it for keeping connection alive (TCP/IP) or WiFi alive. The test case is designed so that the API is not active at that moment (so we know whether or not Whatsapp can indeed use true push rather than poll system). All that the test proved is that Whatsapp can indeed push just like other IM apps when the need arises.

ChMar (correctly I think) outlined some more details which put the exact need for the API in doubt, so I tested this to verify that (separate from the earlier discussion of keeping WiFi alive)

 

[ttsoldier]

Why is this thread still open.

 

[Florin Anghel]

I really tried to follow what the OP said but I came to the conclusion that is not worth it because I was in the following scenarios:
Home with data on - receive notifications from whatsapp
Home with wi-fi on and data on - receive notifications from whatsapp
Roaming (so no data connection) and wi-fi on - receive notification from whatsapp
Roaming no wi-fi - obviously no pushing service.

I must mention that after each time I exit whatsapp, I go over to start a song so that I can kill whatsapp audio api, and I still get notifications afterwards.