Discover more customer stories
Online Gaming Firm Implements Real-Time Analytics and Scales for Planned Growth
BetOnSoft develops and manages more than 110 online casino games, played every day by thousands of players worldwide. The company needed to ensure that its games are highly available, because players are online around the clock. BetOnSoft also wanted to prepare for business growth by scaling its database while maintaining application responsiveness. In addition, its applications must perform key business-critical analytics in real time. In November 2011, the company deployed a hybrid application solution that takes advantage of the high-availability features in Microsoft SQL Server 2012 AlwaysOn and the scalability of SQL Azure. Now, the company’s infrastructure can exceed 10 times its previous peak loads while running intensive real-time data analytics. BetOnSoft has achieved the availability it needs and can use its hybrid infrastructure to scale to meet unexpected business growth.
"SQL Server 2012 Enterprise with AlwaysOn gives us exactly the performance we need. We can exceed 10 times the previous peak game-play load.... and still run intensive analytics in real time."
BetOnSoft, an international gaming software provider with presence in 11 countries, is a fast-growing developer of popular online casino games used by players around the world. The company provides gaming software and hardware infrastructure to independent operators that market and brand the games. Over the past two years, the company has launched new operators into the marketplace and acquired existing operators from other software providers through its superior platform and products.
Currently, BetOnSoft offers more than 110 single-player online games, including slot machines, roulette, blackjack, and craps. These games can be played on computers or mobile devices.
As a growing player in the e-gaming software market, BetOnSoft needs to maintain high availability for its mission-critical gaming applications in order to achieve business success. Their operators market to an international player base, and so there are always players online, 24 hours a day, seven days a week.
Availability was sometimes challenging because when BetOnSoft database administrators would run intensive maintenance operations such as checking the database for corruption, application timeouts would often occur. Additionally, when the company would deploy new software, administrators sometimes had to take the application server offline.
To be competitive, BetOnSoft must be able to be agile and innovative in its technology approach, so it can handle rapid growth in the number of users playing its games. In fact, as the number of operators using BetOnSoft services increases, it is likely that aggressive marketing on any given day would create sudden high demand. To handle such scenarios, BetOnSoft needs the ability to rapidly scale up or down. “We have more than doubled the number of operators in the last 12 months,” says Thomas Pullen, Database Administrator, BetOnSoft. “And our expectations are that we will continue to grow. We needed to make sure that our database software and servers had the capacity to scale rapidly.”
BetOnSoft also sought to out-innovate its competitors by implementing rich functionality for operators and players alike, much of which depends on complex data analysis to produce results in real-time.
To increase availability, scalability, and performance for its multi-terabyte database, in early 2011 BetOnSoft decided to implement a new technology solution.
BetOnSoft began deploying a new solution in July 2011, when it upgraded its database servers and database software. At that time, the company implemented Microsoft SQL Server 2008 R2 x64 Enterprise data management software on two Dell PowerEdge R810 server computers with four 8-core processors and 256 megabytes of RAM.
Each server contains a 640-gigabyte and a 1.2-terabyte memory card made by Fusion-io, a storage-memory company based in Salt Lake City, Utah. Fusion-io memory cards can improve processing capabilities in a data center by relocating active data from centralized storage to the server where it is being processed. This can help reduce latency while also increasing data-center efficiency.
In late 2011, BetOnSoft decided to upgrade further to Microsoft SQL Server 2012 Enterprise. “We had been very happy with SQL Server 2008 R2 overall,” says Pullen. “But we saw features in SQL Server 2012 that we knew would help us with availability, scalability, and performance.”
One of those features is SQL Server 2012 AlwaysOn, a new high-availability and disaster-recovery solution through which customers can query data in replica databases and conduct backup operations from those replicas. AlwaysOn includes availability groups that support a failover environment for a set of user databases that fail over collectively. This feature also includes the AlwaysOn availability group listener, which contributes to easier application server configuration and redundancy.
Additionally, AlwaysOn provides readable database mirror capabilities. The replica databases provide read-only access for use in reporting and backup, which serves to offload some of the primary server’s workload.
“SQL Server 2012 AlwaysOn was the key driver for us,” Pullen says. “Between the availability groups, the readable mirror for offloading reporting and database checking, and the listener, we knew we would increase our availability with SQL Server 2012.”
“Typical data architectures for e-commerce applications involve a high-throughput online transaction processing (OLTP) database from which data is fed into a downstream data warehouse,” says Devan Govender, Chief Software Architect, BetOnSoft. “Data analysis is then usually run on the warehouse, which can be several seconds or even minutes behind. Even small delays are not tolerable in the market-leading gaming products we are building.”
BetOnSoft architected its applications around SQL Server 2012 AlwaysOn, Fusion-io storage, and strategic hardware and network configurations to take advantage of the benefits provided by this platform and achieve its product and performance goals. Govender says, “SQL Server 2012 AlwaysOn is a key part of our solution to achieve real-time results.”
Testing the Solution in the Lab
Prior to implementing SQL Server 2012, BetOnSoft held two series of testing engagements in Oxford, UK in April and May 2011. Then, in September 2011, BetOnSoft worked with Microsoft to test SQL Server 2012 in a production environment at a laboratory session at Microsoft headquarters in Redmond, Washington.
The goal of the Redmond lab was to validate that the technology could support at least 10 times current gameplay workload at BetOnSoft, while still able to perform intensive data analytics in real-time. “We wanted to make sure that the technology could give us an extra level of availability without any performance penalty to the players,” says Pullen. “And, ultimately, we wanted to make sure that the solution supported future scale-up throughput requirements that fell within our acceptable application response times.”
During the lab, BetOnSoft installed SQL Server 2012 instances on each server, activated the AlwaysOn features and set up availability groups and synchronous secondary instances while activating reporting from a readable database mirror. It also conducted failover testing. “We really wanted to run a stress test on the availability groups,” states Pullen. “We were driving SQL Server 2012 to exceed 10 times our peak production load.”
BetOnSoft went live with the new solution in November 2011.
Creating a Hybrid Cloud Solution
BetOnSoft is also running several critical services on Microsoft SQL Azure, a cloud-based data-storage environment that provides high availability by storing multiple database copies and providing fast provisioning.
For example, BetOnSoft maintains its error-reporting service in the Windows Azure cloud with data stored in SQL Azure databases. This service monitors, by geographic region, the number of players worldwide that are experiencing problems launching or playing games on their computers. “For some services, such as error reporting, it makes sense to manage that outside the data center,” says Govender. “For example, there could be issues with the data center that make it inaccessible for error reporting.”
BetOnSoft also runs certain marketing applications on Windows Azure, where demand can spike as a result of campaigns run by marketing partners. “It was a no-brainer for us to run services that have unpredictable demand in the cloud,” says Govender, “We scale up to meet demand and back down when demand subsides.”
Another Windows Azure service is used to collect statistics on the quality of connections to the company’s games. Statistics are collected for download rates, latency, and number of connection errors.
BetOnSoft also has a Windows Azure monitoring service that collects data on transaction rates, the number of games played, and other activity metrics in a SQL Azure database. It constantly analyses these metrics to detect and send alerts about any anomalies that require attention.
With the new SQL Server 2012 solution, BetOnSoft can process more than 10 times its previous peak workload while running real-time data analysis. The solution also increases availability and gives BetOnSoft the capacity to scale for growth. Additionally, the company has easier IT administration and can provide better service to its operators.
Processes 10 Times Previous Workload While Running Real-Time Analysis
During lab testing, BetOnSoft were able to exceed their target of 10 times their current production workload. “SQL Server 2012 Enterprise with AlwaysOn gives us exactly the performance we need,” says Pullen. “We can sustain more than 10 times the current peak game-play load and still run intensive analytics in real-time.”
This performance is aided by the Fusion-io memory card, which contributes to low database latency because it does not rely on SAN storage. “Using local attached storage helps BetOnSoft get the throughput it needs with SQL Server 2012,” says Josh Miner, Director of Product Marketing, Fusion-io. “With reduced latency, the server computers get data faster and can process that data hundreds of times per millisecond. That contributes to faster and more consistent response times for BetOnSoft game players.”
Increases Availability for Mission-Critical Applications
SQL Server 2012 AlwaysOn gives BetOnSoft the enterprise-level robustness it needs to ensure high availability for the company’s mission-critical online gaming applications. “Before we upgraded our servers and implemented SQL Server 2012, I could not regularly check the database. Whenever I did, it would cause application timeouts,” says Pullen. “Now, with the high availability we get from SQL Server 2012 AlwaysOn, I can check the database every week, and I can be confident that the database is corruption free.” BetOnSoft also checked automatic failover time during testing. “We were prepared to accept a time of two minutes, and it only took 14 seconds, which was a huge win for us,” says Pullen.
Taking advantage of AlwaysOn availability groups, BetOnSoft can also deploy new game features faster than before. “With the availability group listener, for example, multiple application servers can be configured identically, no matter where the database is running,” says Pullen. “That further increases availability and helps us avoid downtime when we deploy new software.”
The company also ensures high availability from its Windows Azure monitoring services. “We’re using SQL Azure for our core monitoring services, and it helps us ensure the highest availability for our critical services,” says Govender.
SQL Azure also enhances security for those services. “The firewall and security configurations in SQL Azure are great,” says Govender. “Our cloud-based services are now as secure as our data-center-based services.”
Provides Hybrid IT Structure to Accommodate Scalability
When BetOnSoft tested its new solution prior to going live, it confirmed that SQL Server 2012 could sustain the level of throughput needed to meet future business growth. “We now have the capacity to add a lot more players and operators while not losing any application responsiveness,” says Pullen. “If our customer base grows by 10 times, we know we’ll still have great performance with SQL Server 2012 AlwaysOn.”
And with Windows Azure, BetOnSoft has an added layer that it can use to scale to handle unexpected high demand for its services. With that capability, BetOnSoft can better compete in the online gaming marketplace. “In our business, responsiveness and scalability are very important, because we need to retain the same, fast application performance while more players are playing the games,” Pullen says. “We want to grow our business, and SQL Server 2012 positions us to do that.”
SQL Server 2012 features like AlwaysOn availability groups and the availability groups listener, which support easier server configuration and failover management capabilities, will help simplify administration for BetOnSoft database administrators. “Using the readable mirror in SQL Server 2012, I can check the database frequently and easily, as well as offload reporting,” says Pullen. “That really reduces time and effort for me, making my job easier from a management perspective.”
Enhances Service to Operators
In addition, SQL Server 2012 helped BetOnSoft enhance the services it provides to the operators that run the company’s games. For instance, BetOnSoft is now able to provide fraud detection, VIP identification, and marketing campaign analysis services to operators in real time. “Having the SQL Server 2012 readable database mirror functionality makes this easier for us to do, because we can provide access to reporting data without compromising the primary server,” says Pullen.
With its SQL Azure–based services, the company can use a leaner infrastructure overall and is also gaining valuable metrics that can be used to improve the user experience. “It’s very valuable for us to see that someone in the United Kingdom is having a great download experience, while a player in India is having a bad one,” says Govender. “It gives us a complete and detailed view of the global player experience.”
The company also can better detect issues and anomalies. “We use our SQL Azure–based monitoring service to see where the problems are and if there are certain trends,” says Govender. “We use these metrics to enhance our services to improve the overall user experience.”
Ultimately, SQL Server 2012 fulfilled all of the company’s requirements during testing, which confirmed that it was the right technology to align with BetOnSoft business goals. “We would not have gone live with SQL Server 2012 if we hadn’t had that success in the testing phase,” says Pullen. “Those results showed us that we were implementing the right technology to meet our business growth and maintain the high availability and strong performance we need to be competitive in online gaming.”