We often hear the question "Which is better OpenSER or SIP Express Router?". Well, now we know because we have completed a thorough and carefully controlled benchmark test of both SIP proxies. For an overview of our test, please see my blog from April 18th (OpenSER Performance Test). The complete test plan and results will be posted on www.transnexus.com soon.
The comparison results for 200 calls per second are:
.............................................CPU .........................Call
.........................................Utilization.............. Completion
OpenSER V1.1 .....................92.9% ......................97.17%
OpenSER V1.2 ....................83.3% .....................100.00%
SER 2.0 ..............................83.1% .....................100.00%
The comparison results for 220 calls per second are:
.............................................CPU .........................Call
........................................Utilization ...............Completion
OpenSER V1.1 .......................NA ...........................NA
OpenSER V1.2 ...................90.5% ......................99.79%
SER 2.0 .............................90.7% ......................93.43%
Conclusions:
- OpenSER V1.2 is significantly better than OpenSER V1.1 in terms of performance, call completion and post dial delay.
- OpenSER V1.2 and SER 2.o are identical in performance, but OpenSER V1.2 performed more reliably as it approached the limit of CPU utilization.
Friday, April 27, 2007
Thursday, April 19, 2007
OpenSER V1.2 is 13.4% faster than V1.1
This post is follow-up on my post yesterday about our performance testing of OpenSER V1.1, so please refer to that post for details on the test setup and test scenario.
We have completed our performance testing of OpenSER V1.2. The comparison results are:
V1.1 - 200 calls per second - 92.9% CPU Utilization - 97.17% call completion
V1.2 - 220 calls per second - 90.9% CPU Utilization - 99.79% call completion
OpenSER V1.2 performance is 13.4% better than V1.1 and most importantly its call completion rate under a heavy load is excellent.
These results indicate that OpenSER V1.2 running on server with two Xeon, dual core CPUs could conservatively handle a 600 million minutes per month of VoIP traffic.
We have completed our performance testing of OpenSER V1.2. The comparison results are:
V1.1 - 200 calls per second - 92.9% CPU Utilization - 97.17% call completion
V1.2 - 220 calls per second - 90.9% CPU Utilization - 99.79% call completion
OpenSER V1.2 performance is 13.4% better than V1.1 and most importantly its call completion rate under a heavy load is excellent.
These results indicate that OpenSER V1.2 running on server with two Xeon, dual core CPUs could conservatively handle a 600 million minutes per month of VoIP traffic.
Wednesday, April 18, 2007
OpenSER Performance Benchmark
We continue to see a growing number of service providers using SIP Express Router (also known as SER) and OpenSER as infrastructure SIP proxies for their carrier operations. SER (www.iptel.org) and OpenSER (www.openser.org) are mature open source projects for a very stable, high performance SIP proxy. SER was the original project and OpenSER is fork of SER started a couple of years ago. Both projects are excellent and OpenSER has growing popularity because it more rapidly introduces new features.
We often hear the question, "What is the capacity of OpenSER?" There have been benchmark tests that show OpenSER and SER can manage thousands of SIP calls per second. But how about a benchmark test that simulates a real production environment with call failures, retries and call detail reporting? Since we have not seen such a test report, we decided to perform our own benchmark test. For our test we configured OpenSER V1.1 on a Dell Precision 490 server with two Intel Xeon 5140 2.33 GHz CPUs and 4 GB of RAM. To reduce the traffic load requirements, we disabled one CPU and one core on the remaining CPU. We ran the test on a single core of a dual core chip. The operating system was CentOS 4.4.
To simulate a production environment, we configured OpenSER to communicate with an OSP server for call routing and CDR collection. For each call there were five possible destinations returned in random order. Four of the five destinations were configured to simulate call setup failures (no TCP response, no SIP response, call rejected and no route). We found that 200 calls per second was the maximum the single core processer could handle. If we had used all four CPU cores we expect the results would have been 800 calls per second. To be conservative, we would recommend service providers to plan on maximum CPU utilization of about 60%. This would establish the OpenSER planning gauideline of 500 calls per second on a server with two, dual core Xeon CPUs.
If you assume 15% of a service provider's traffic occurs during the busy hour, a 50% Answer Seizure Ratio (ASR) and a 3 minute average call duration, then 500 calls per second equates to 540 million minutes of VoIP traffic per month! We think this is impressive for an open source SIP proxy running on a server with a retail price of $2,967. If you want a copy of the test plan and results, send an e-mail to me at jim.dalton@transnexus.com.
We will run the same benchmark test on OpenSER V1.2 and then on SER to get a clear comparison of the different releases. When we are done, we will publish the results on www.transnexus.com.
We often hear the question, "What is the capacity of OpenSER?" There have been benchmark tests that show OpenSER and SER can manage thousands of SIP calls per second. But how about a benchmark test that simulates a real production environment with call failures, retries and call detail reporting? Since we have not seen such a test report, we decided to perform our own benchmark test. For our test we configured OpenSER V1.1 on a Dell Precision 490 server with two Intel Xeon 5140 2.33 GHz CPUs and 4 GB of RAM. To reduce the traffic load requirements, we disabled one CPU and one core on the remaining CPU. We ran the test on a single core of a dual core chip. The operating system was CentOS 4.4.
To simulate a production environment, we configured OpenSER to communicate with an OSP server for call routing and CDR collection. For each call there were five possible destinations returned in random order. Four of the five destinations were configured to simulate call setup failures (no TCP response, no SIP response, call rejected and no route). We found that 200 calls per second was the maximum the single core processer could handle. If we had used all four CPU cores we expect the results would have been 800 calls per second. To be conservative, we would recommend service providers to plan on maximum CPU utilization of about 60%. This would establish the OpenSER planning gauideline of 500 calls per second on a server with two, dual core Xeon CPUs.
If you assume 15% of a service provider's traffic occurs during the busy hour, a 50% Answer Seizure Ratio (ASR) and a 3 minute average call duration, then 500 calls per second equates to 540 million minutes of VoIP traffic per month! We think this is impressive for an open source SIP proxy running on a server with a retail price of $2,967. If you want a copy of the test plan and results, send an e-mail to me at jim.dalton@transnexus.com.
We will run the same benchmark test on OpenSER V1.2 and then on SER to get a clear comparison of the different releases. When we are done, we will publish the results on www.transnexus.com.
Thursday, April 5, 2007
Secure VoIP Peering
At the Spring VON show in San Jose, CA I had the opportunity to give a presentation on Secure VoIP Peering. The presentation is an overview tutorial on how standard Public Key Infrastructure (PKI) services and the OSP protocol are being used today to enable secure peer to peer VoIP calls.
For those who do not know, OSP is a protocol standard defined by ETSI, the European Telecommunications Standards Institute. ETSI is best known as the standards body that created the global mobile phone standard known as GSM. OSP is a protocol used by a softswitch or session border controller to query and external route server for call routing and then also report call detail records when the call is finished. OSP is a set of data rich XML meesages transported over HTTP so it is a natural fit with SIP and web based services. Further, OSP is designed for routing, authorization, pricing and usage reporting for any IP service, not just VoIP.
To view the Flash version of the Secure VoIP Peering presentation, go to http://www.transnexus.com/Flash/2007_Spring_VON_-_Secure_VoIP_Peering/player.html
For those who do not know, OSP is a protocol standard defined by ETSI, the European Telecommunications Standards Institute. ETSI is best known as the standards body that created the global mobile phone standard known as GSM. OSP is a protocol used by a softswitch or session border controller to query and external route server for call routing and then also report call detail records when the call is finished. OSP is a set of data rich XML meesages transported over HTTP so it is a natural fit with SIP and web based services. Further, OSP is designed for routing, authorization, pricing and usage reporting for any IP service, not just VoIP.
To view the Flash version of the Secure VoIP Peering presentation, go to http://www.transnexus.com/Flash/2007_Spring_VON_-_Secure_VoIP_Peering/player.html
Wednesday, April 4, 2007
VoIP Peering Panel at Spring VON
Spring VON in San Jose was a very good show - the VoIP market is looking very positive. It has taken me two weeks to follow-up on actions items from customer prospects and get back to routine matters such as blogging.
One of the VON sessions I found most interesting was the panel discussion on IP Voice Peering. The panelists came from a good cross section of companies (iBasis, NeuStar, Ag Projects, XConnect and Arbinet) and each had there own unique perspective on peering. Some of the interesting points from the discussion were:
1. Public ENUM is dead. Many in the industry have pushed for the idea that all telephone numbers with SIP access should be included in a public ENUM directory to enable peer to peer calling. This is an appealing idea for VoIP users because it would eliminate telephone companies and enable global peer to peer VoIP calling free of telecom access charges. While the public ENUM concept makes sense, there are too many political and market sources determined to keep it from happening. Perhaps the panel agreed Public ENUM is dead because it is a threat to their business.
2. The SPIDER Registry. No one wants to share their numbers. While everyone in the VoIP world complains that telcos are "Walled Gardens", or closed networks, the reality is that VoIP service providers are just as proprietary with their networks and customers. VoIP service providers do not want to share the telehone numbers of their customers with any third party. So how to you get broader VoIP peering adoptance if VoIP service providers will not disclose their telephone numbers? One well thought out solution appears to be the non-profit SPIDER registery created by Arbinet. VoIP service providers can list their numbers in the SPIDER registry and still maintain control over who can access those numbers. The SPIDER registry acts as a central provisioning mechanism for making VoIP numbers available to selected peering partners. The idea is good, but trust is still an issue. As expected, all the panelists were SPIDER sceptics since it has been created by Arbinet.
3. VoIP peering depends on the "call context". This an interesting new term that means call routing, or peering, depends on who is calling who and when. Ah-ha, so VoIP peering is not just as simple as connecting to VoIP users. VoIP peering depends on a variety of issues such as competition between peers, accounting and settlement, and technical interoperability. The panel agreed that one of the great drawbacks of ENUM is that it cannot convey "call context". ENUM only conveys the called number. To determine peering partners for a call, the calling and called numbers, source IP address, VoIP protocol, service type and many other factors must be known. As the VoIP peering market develops, we expect implementations of the OSP supported VoIP peering will grow since this well established ETSI protocol already addresses all of these "call context" requirements.
One of the VON sessions I found most interesting was the panel discussion on IP Voice Peering. The panelists came from a good cross section of companies (iBasis, NeuStar, Ag Projects, XConnect and Arbinet) and each had there own unique perspective on peering. Some of the interesting points from the discussion were:
1. Public ENUM is dead. Many in the industry have pushed for the idea that all telephone numbers with SIP access should be included in a public ENUM directory to enable peer to peer calling. This is an appealing idea for VoIP users because it would eliminate telephone companies and enable global peer to peer VoIP calling free of telecom access charges. While the public ENUM concept makes sense, there are too many political and market sources determined to keep it from happening. Perhaps the panel agreed Public ENUM is dead because it is a threat to their business.
2. The SPIDER Registry. No one wants to share their numbers. While everyone in the VoIP world complains that telcos are "Walled Gardens", or closed networks, the reality is that VoIP service providers are just as proprietary with their networks and customers. VoIP service providers do not want to share the telehone numbers of their customers with any third party. So how to you get broader VoIP peering adoptance if VoIP service providers will not disclose their telephone numbers? One well thought out solution appears to be the non-profit SPIDER registery created by Arbinet. VoIP service providers can list their numbers in the SPIDER registry and still maintain control over who can access those numbers. The SPIDER registry acts as a central provisioning mechanism for making VoIP numbers available to selected peering partners. The idea is good, but trust is still an issue. As expected, all the panelists were SPIDER sceptics since it has been created by Arbinet.
3. VoIP peering depends on the "call context". This an interesting new term that means call routing, or peering, depends on who is calling who and when. Ah-ha, so VoIP peering is not just as simple as connecting to VoIP users. VoIP peering depends on a variety of issues such as competition between peers, accounting and settlement, and technical interoperability. The panel agreed that one of the great drawbacks of ENUM is that it cannot convey "call context". ENUM only conveys the called number. To determine peering partners for a call, the calling and called numbers, source IP address, VoIP protocol, service type and many other factors must be known. As the VoIP peering market develops, we expect implementations of the OSP supported VoIP peering will grow since this well established ETSI protocol already addresses all of these "call context" requirements.
Subscribe to:
Posts (Atom)
