Scaling a 24-hour online WordPress marathon on Kyup’s infrastructure
Who are we?
SiteGround is a web hosting provider with more than a 10-year history. We are one of the leading service providers offering managed WordPress services, among others. We are also active supporters of the WordPress community and there is hardly a WordPress event that goes without a visit from at least one of our team members. That’s why when we were contacted by the organisers of WordSesh, the annual 24-hour online conference in the community, we were thrilled to accept the challenge.
What is WordSesh?
WordSesh is a 24-hour WordPress conference that gathers the best speakers in the community from all over the world into a free non-stop WordPress marathon. What’s more, WordSesh is an interactive event - the attendees tuning in are not just passive viewers while there’s a session in progress, the users who are online are chatting, asking questions, making a point - they are all active participants in a virtual WordPress camp. Staying connected and being engaged is part of the beauty of the community.
Apart from the excitement to be involved in such an awesome project, it was very interesting from a technical point of view too, since we needed to ensure that thousands of visitors will be able to follow the free live stream for 24 hours with no interruption or other technical issues.
The event was to take place on December 22nd, just two days before the winter holidays. This was its third edition and the organisers expected it to be their most popular so far. This meant thousands of viewers at any point in time. For the 24-hour duration of the event, the WordSesh website would experience enormous and hard to predict traffic surges, as viewers from all over the world joined in to stream the current session.
The event is happening entirely online, so it was crucial that all traffic spikes were handled seamlessly. Even the shortest service interruption could result in a major failure - a 10-minute downtime on a medium website with steady traffic throughout the year might go unnoticed, imagine how detrimental even a 60-second downtime would be, should it occur, say, during the most visited session of the event. Like going to the movies and having your movie cut off during the most interesting part.
Why we chose Kyup
When the WordSesh team reached out to us, Kyup container cloud platform had recently been launched as a separate project by the founders of SiteGround and it came as a great solution for the challenges of the project.
Our usual and more traditional approach to large-scale projects was simply not flexible enough in this case. It would have involved setting up several dedicated servers, including a few idle ones, ready to be used as load balancers in case of an upsurge in traffic. Having all the servers ready would have required a lot of time for the hardware to be ordered, delivered and pre-installed. Even if time was not of the essence for the initial setup, it meant that if we needed more machines urgently, it would have been practically impossible to add them on the fly. So we needed to make a considerable preliminary investment in hardware in order to be on the safe side with this setup. What is more, no matter how much spare hardware we had, there would have always been a risk for something unexpected to happen. Even if the system needed just a few more resources for a very small amount of time, we would not have been able to provide those instantly with this setup.
At the same time, the new Kyup container infrastructure seemed to overrule all the drawbacks of the dedicated server setup. First, Kyup containers were very fast to deploy: it literally takes seconds to spin up a new container instance. This significantly shortened the initial setup time. Additionally, Kyup containers were extremely easy to scale both vertically and horizontally - and without service interruption. No reboots were needed to add more RAM or CPU cores and a new container could be launched in seconds to balance the load.
That is why we went with the Kyup infrastructure: it was fast to deploy, flexible and resource-efficient.
Our WordPress experts worked together with Kyup’s system architects to build a redundant architecture tailored to WordSesh’s specific use case from the start.
We started out with six Linux containers. Two load-balancing ones were set to distribute the incoming connections to two main PHP FPM containers and two MySQL instances in the back. We kept pre-configured images of each of those instances, so that we could add new ones in seconds if high load was detected. Since Kyup containers have a 5-second provisioning, we could literally plug those in instantly.
We enabled caching for the static resources and configured that to work with a CDN by CloudFlare. The pair of PHP FPM containers also had opcode caching and Memcached enabled. We used Stale cache to regenerate the dynamic cached content every 10 seconds with a single connection to the PHP containers, since the main challenge when implementing a dynamic caching system is purging the cache whenever there’s a change on the site.
Right behind the PHP FPM containers, there were two MySQL nodes configured with a master/slave replication. Additionally, each instance was residing on a different host node as an extra precaution. We configured the whole environment with a focus on redundancy to make sure there was no risk of any downtime whatsoever.
This whole system worked so well that the initial setup of two main containers, two load balancers and two database instances easily handled all the traffic and fluctuations coming our way with no need to add any additional hardware. The event ran seamlessly for the whole 24 hours no matter how many visitors were tuning in to stream it simultaneously or from which part of world.
SiteGround <3 Kyup
At the end of the day, WordSesh 3 was visited by more than 3000 unique visitors from 89 countries with hundreds of people streaming and interacting at a time - which makes it one of the largest WordPress events of 2014. Powered by the scalable, redundant and resource-efficient architecture that SiteGround built on top of Kyup containers, the event ran perfectly, setting the bar even higher for next year.