The modern world offers many methods of communication, right at our fingertips. Many of these have given a renewed emphasis to written communication: Society has shifted from letters and telegrams to telephony, voice calls, and a more direct form of communication, only to find technology pulling us back to the written word. Whether it is social media, email, or a range of dedicated messaging services, much of modern communication results in reading and typing rather than listening and speaking.
However, voice calls remain critical to the way we keep each other informed in both business and our personal lives. Today, many of these calls take place via the internet using VoIP services instead of a telephone line. From long conversations for catching up with friends or briefing colleagues, taking a call to discuss sensitive issues, or maximizing your time by taking calls during the drive to work, voice calls aren’t going anywhere.
You only have to navigate customer services on a company’s website to realize how much you miss talking on the phone with a real human being on the other end. What starts as a simple problem ends with unhelpful FAQs, customer service chatbot loops, and slow-to-reply messaging services.
Nothing can beat getting on a call and ironing out a problem right there and then. In contrast, written communication means waiting for responses, dragging people’s attention to your message to get their input, and often losing sight of the original intent in a sea of messages sailing back and forth.
However, for voice calls to create a personal and effective interaction, service providers need to ensure a great user experience. Unfortunately, one of the most frustrating problems affecting voice calls today is one-way audio, which is what we are going to focus on in this article.
One-way audio; You only miss something when it’s gone
We’ve all experienced it: you get on a call only to hear nothing but silence, or you pick up the phone and end up shouting at someone who can’t hear a word you’re saying. This is the result of one-way audio, one of the trickiest problems service providers face.
The audio is missing, and finding something that isn’t where it’s meant to be can be challenging. The first step is to understand the potential causes of the problem, and while many possible issues can produce one-way audio, in most cases, there are four main culprits.
Four primary causes of one-way audio
1. Missing RTP stream – The RTP stream containing the media (audio from one side of the call) is not being delivered to one or more parties. This can be due to a misconfigured firewall or other network elements such as the router, software bugs, or the packets exceeding the maximum hop count.
2. Silent payload – The packets make it to the intended recipient but contain nothing but silence. Often, silent payloads result from wrongly configured media gateways or bugs in the gateway or other sender software. Such bugs can also cause cross-talk or ghost voices instead of pure silence.
3. Multiple streams – Sometimes referred to as zombie streams, multiple streams happen when the receiver gets more than one RTP stream, and the system has to choose which one makes it to the end-user. Often the unwanted zombie stream contains nothing but silence; when chosen, the receiving party only hears silence as the actual media is discarded.
4. Irregular RTP stream – If the RTP stream received shows unexpected properties or has some irregularities, the receiver may ignore these packets, producing only silence. This can happen when a problem on the sender’s side causes the RTP stream to reset. Each packet is labeled by a sequence number which increases throughout the call. If the stream resets, this sequence number starts again from zero. Therefore, if the receiver (who expects labeled packets in an increasing series) suddenly receives packet zero, they may choose to discard the media, resulting in one-way audio.
Real-life one-way audio case studies to learn from
So, what do these sorts of one-way audio issues look like in real life? Below are two case studies from our clients to demonstrate how they might occur.
Case study #1: Zombie Stream
An enterprise client got in touch with us regarding a recurring one-way audio issue they were having. With no obvious pattern to spot the cause, the client reported each incident as it occurred.
Initially, the problem happened more than once a day on average. After investigating we found that the problem was much more prevalent, affecting upwards of 100x more calls than the reported level.
The one-way audio was the result of a bug in the Cisco Call Manager (CUCM) that meant specific calls continued sending RTP packets after the call has ended. This RTP stream continued until the Call Manager restarted.
As this zombie RTP data stream was refusing to die, it created a one-way audio issue if an unlucky user started a new call with the same destination IP/port as the new call stream. This made the problem difficult to spot without a detailed investigation of RTP traffic.
In fact, before Qrystal got involved, teams from different vendors spent several months investigating the case without success. Using Qrystal the client was able to identify zombie streams as the cause, isolate the sending device and notify the vendor in less than a week, thereby reducing the time to fix for one-way audio by about 80%.
Case study #2: Two Swedes in Germany
Consider two Swedish travelers in Germany wishing to call each other, and no audio making it through (silent call). Both Swedes were customers of the same Swedish carrier and the same roaming network in Germany. No issues were present from a signaling perspective, but the problem was hiding in the RTP data’s long path between the call parties.
Packets from the caller had to make many separate trips:
- Caller to the German carrier
- German carrier to the German international service
- German international service to the Swedish carrier
- Swedish carrier back to the German international service
- German international service back to the German carrier
- German carrier back to the callee
After investigating, we found that the data was being lost between measurement points 4 and 5 (within the German international service) due to the route being longer than the default TTL/hop count. Unfortunately, this meant the router had no other option but to discard the packets. The default TTL should generally be increased from 64 to 128 to ensure good service when dealing with international calls.
Finding the right tools to help
While understanding the causes of one-way audio is an important first step, it doesn’t actually solve the problem. Searching through the data and resolving one-way audio issues is no easy feat. It takes considerable experience just to know what you’re looking for.
Thankfully, there are now tools available that simplify the processes needed to identify and rectify one-way audio problems, saving service providers significant time.
More basic tools can help you dig through traces to solve issues, but the real, game-changing tools monitor all RTP streams to check quality across your entire network and automatically detect potential causes of one-way audio. This could entail tracking statistics and metrics such as sender restarts or other tell-tale signs of one-way audio. With the right tools on your side, you can help qualify the root cause of one-way audio while also quantifying its impact on the end-users.
Qrystal is an example of a tool just for that –- providing analysis on voice packets to ensure high-quality voice services and happy customers. With automated troubleshooting and state-of-the-art analytics, you receive the meaningful insights that directly lead to improved one-way audio solutions.
Conclusion
One-way audio is a severe concern to all voice service providers. In today’s world, where customers can quickly share their experiences of unreliable service, offering the best possible experience is more important than ever. Thankfully, with the right tools such as Qrystal, you get comprehensive visibility and packet-by-packet inspection to quickly and methodically spot potential causes of one-way audio and get to work towards a solution.
