Multi-Gaming Community

Mumble 1.2.4

Today we've updated the Mumble server to release 1.2.4.

For a full version of the changelog please visit the Mumble website.

The mayor highlights of this update are:

  • Opus is now the primary audio codec of choice for Mumble.
  • Positional audio for Source engine based games

You can download the new mumble client either through the update notification of your current mumble installation, or visit www.mumble.info.

Win a copy of Red Orchestra 2 Rising storm, play some MC

As announced on our forums, we'll be giving away 4 copies of Red Orchestra 2: Rising storm to the 4 winners of the Minecraft UHC competitions.

UHC is a gamemode made popular my the mindcrack community. It consists of players being randomly spawned around a 2000x2000 map leaving them to survive and battle it out to determine the winner. Everything is against you here, as your health does not regen naturally. The only way you can achieve this is by eating golden apples or drinking regeneration potions.

Sound easy enough eh? Well your wrong there.

Golden apples now require 8 gold ingots rather than nuggets, along with the challenge of getting an apple from a tree. Glistering melons now also require a gold block rather than a gold ingot. Also as a sadistic bonus feature players can wrap a fellow players head in gold ingots(as you would do with an apple) to create an item that will regen you health.

For more information on the rules, game details and signups, visit the UHC topic here.

Part six: Migrating MrWhite (1 of 2)

Unfortunately we did not succeed yet in completing our migration of MrWhite. Primarily this is because the amount of work turned out to be more than we anticipated. We don’t want to rush things so we take our time to set everything up and tweak it accordingly.

Currently the base system (Debian 7) is installed and we’ve moved over most files, websites, databases etc.

We’ve tweaked, optimized and corrected a lot of issues in the MySQL databases. This should speed up the time required to run queries and send back the data.

On the webservers side we’ve implemented “Varnish”, a reversed caching proxy. Varnish will cache all pages that our users visit into the memory of the server, so they can be loaded up much faster afterwards. Websites we run on MrWhite require some tweaking as well to make optimal use of the new caching features. For example on Drupal, our CMS for spa.net we’ve enabled special modules that can feed Varnish with optimal data for caching operations. A few quick tests already show the benefits of the new setup. Sites like our forums and HLstatsX almost load instantly when being accessed.

While enabling the varnish module for Drupal we noticed our Drupal install is out of date and so are a lot of modules. We directly took the liberty of updating them to the latest standards to prevent any security issues.

For our IRC bouncers we decided on a clean install, rather than a backup/restore operation. Our ZNC software used on the old system is quite old, so a new, from scratch installation would make for a much cleaner/nicer install. A little note to our users who have a ZNC account, you’ll be receiving new account information once the server goes live.

Next week will be a busy week for us, so most likely we’ll be finishing up the migration in a week or 2.

In the next part we’ll do some rambling about why you shouldn’t number your game servers and where game developers have gone wrong the last couple of years in regards to community building.

Part five: Introducing the VM’s

When we created SpA we got ourselves 2 machines. As it’s a good tradition in the IT business to name your machines. We've chosen to name our machines after characters from the film “reservoir dogs”.

We named our machines “Mr. White” and “Mr. Blonde”. These machines kept being replaced throughout the years, but always kept the same names. If I am not mistaking the current machines we have are the fourth generation of MrWhite’s and MrBlonde’s.

Later on when we decided to virtualize the machines we kept the names the same for the first VM’s we created. Hence the confusing part where we have hardware platforms MrWhite and MrBlonde, and Virtual machines MrWhite and MrBlonde.

MrWhite got virtualized a few years ago and that machine got an extra VM running on it providing us with an Exchange mail environment and Minecraft freebuild space. This machine was named “Mr. Green”. Not a character in the film, but a playful reference to the retarded amounts of weed that was consumed by our members, including myself.

After the rebuild last few weeks the current setup looks like this:

Hardware platform MrWhite’s VM’s:

  • MrWhite, running all our non-gaming services, such as the websites, stats, databases, mumble, IRC bouncers, IRC bot and everything else I forgot. This VM is equipped with 4 CPU cores, 8GB memory and currently 100GB HDD space on our SAS drives. This VM is getting a full rebuild this weekend as it will be allocated on the new SSD and it will get rebuild under the supervision of a Linux expert. The OS running in this VM currently is Debian 6.0 and will be Debian 7.0 after the reinstall.
  •  MrGreen, running Exchange mail environment. This VM is allocated with 2 CPU cores, 6GB memory and 100GB HDD space. This VM most likely is going to be reinstalled in favor of running “OpenExchange” on Linux, the opensource counterpart of Microsoft’s exchange. Currently the OS running in this VM is Windows 2008 R2.
  •  MrBlue, running our MvM, events, bball2 and vanilla TF2 servers. This VM is equipped with 4 CPU cores, 4GB memory and 40GB HDD space. The OS running in this VM is Debian 7.0

Hardware platform MrBlonde’s VM’s:

  • MrBlonde, currenly only responsible for storing the backups of all other VM’s, primarily MrWhite’s data. The VM is equipped with 2 CPU cores, 4GB memory and 240GB HDD space on the RAID setup. The OS running in this VM is Debian 7.0
  • MrBrown, running 2 “Knights and Merchants” servers. One server is a public normal server, the other server run’s KaM’s development releases. This server is fully assigned to the developers of KaM to provide a powerful platform to test their development releases on. The VM is requipped with 1 CPU core, 2GB memory and 40GB HDD space. The OS running in this VM is Debian 7.0.
  • MrOrange, running 3 64 slots “Red Orchestra: Rising storm” servers. This is the latest game we’ve adapted to the community and still looking how this one will work out. This game is with no doubt the most resource hungry of them all in terms of CPU usage due to the high slot capacity. This VM is equipped with 3 CPU cores, 8GB memory and 40GB HDD space. The OS running in this VM is Windows 2008 R2 as the game doesn’t support being ran at Linux.
  • And last, but not least, MrPink, running all Minecraft servers. This VM is the most memory hungry. Equipped with 2 CPU cores, 16GB memory and 100GB HDD space. The OS running in this VM is once again Debian 7.0

As you could read, this weekend we’re reinstalling the VM MrWhite, so the next article will be written after the weekend and hopefully published on the website running on the new VM.

Part four: Why VMware?

As written earlier, one of the main reasons why we chose to use VMware to virtualize our machines is that we can give each game its own OS/platform which allows for easier access/privilege management for the various people in SpA taking care of the games.

The other major reason is scalability. As everyone knows with games, the arrive, they become popular and after a while they slowly die out again. Because of this we had great difficulties dividing our available resources across the machines. Games are tied to the IP address they run on, and if we wanted to move game X to machine Y we’d had to move its IP as well. However, most of the time the same IP was used for other game servers as well, posing a problem.

With VMware now we gave each OS its own IP address. Per VM we can assign an amount of resources that machine could use. Due to this, if a game suddenly becomes more popular than we expected we can easily assign more CPU cores, memory and hard drive space to a VM. Likewise we can remove resources from a VM when the player count of the game in question goes downwards again.

Because the servers are virtualized now it makes it very easy to move VM’s between the 2 hardware platforms. Games becoming less popular and requiring less resources can be moved off to the older MrWhite. Games that require a lot of resources and handle a lot of players can be moved to newer more powerful machine MrBlonde. This all can be done with minimal downtime and in a matter of minutes.

Another upside of VMware is that we can login to the console of the operating systems, as if you were sitting at the datacenter with a mouse, keyboard and monitor attached. This way if we somehow mess up something at a machine, let’s say a bugged kernel or broken windows update, we don’t need to drive over to the datacenter to fix it. We can now simply login to the VMware console and remotely take over the screen and other input devices and fix any OS problems we encounter.

At SpA we’re known to mess stuff up every now and then. By limiting the resources per VM we can mess up as much as we want and only affect the game platform we’re working on at that moment. All the other games won’t notice a thing of the problems, where in the past if we had an issue, all games on the machine would notice that.

In part 5 we’ll take a trip past all the new VM’s, their names and specifications as well as the services they run.

Part three: The datacenter trips (2 of 2)

The new SSD had arrived so it was time for our second trip to the datacenter. Armed with an external USB backup drive and the new SSD we went on our way.

As we had setup the MrBlonde VM with so called “thin provisioning” we thought it would be a quick process to make a backup from the VM. After all 100GB was provisioned for that VM, but only 20 or 30GB was actually used. Turns out when you want to create a backup from a thin provisioned VM, it would still download the full 100GB. The trip was already going to take longer than expected…

First thing we did was staring the backup of the MrBlonde VM to the external HDD, so in the meantime we could start rebuilding the hardware machine MrBlonde.

As we require more hard drive space in MrBlonde to be able to fully backup MrWhite (MrWhite previously had 100GB space, where with the new SSD it will get 240GB allocated), we had to create a RAID in MrBlonde. MrBlonde currently has 3 130GB Cheetah SAS drives in its system, so we had to RAID0 2 drives to get the minimum capacity of 240GB we required.

The RAID was setup and it was time to install VMware ESXi 5.1 on MrBlonde. After booting the VMware installer it was time to select the hard drive it should be installed on. To our surprise VMware didn’t show *any* available hardware space at all. After some research we found out that in ESXi 5.1 our 3ware RAID controller no longer officially was supported.

After doing quite some research on the internet we figured out we could “slipstream” the 3ware drivers into the VMware installer to make it able to work with our 3ware controller again. Luckily for us some open source developers had been so kind to create a tool to easily slipstream vib driver files into a VMware ESXi installer ISO (needless to say we’ve send a donation their way for the hours of work they saved us).

After creating the new ISO, loading it up on a USB drive and booting the system, VMware finally saw our 3ware controller again and we could go ahead with the install!

Once VMware was installed we ran into our next surprise. The machine has 2 NIC’s, both with a different chipset. One was recognized, the other wasn’t. As we just had some experience with the vib driver files this was a quick fix by downloading some extra drivers and inserting them into the system (We use the second NIC for our internal management LAN).

MrBlonde was fully installed and operational again, however, our backup to transfer the MrBlonde VM was still going. As we were in the datacenter already for like 3 hours we didn’t really want to wait for this.

After searching around for a while on google we found out that there was something called “ovftool”, a VMware tool to migrate VM’s with from one host to another (We don’t have vCenter so we couldn’t do it the ‘normal’ way). The downside however of ovftool was that it transfers the VM through the host running the tool. The upside though, it respected thin provisioning and would only transfer the actual data.

After thinking for a minute we realized that we could simple start ovftool in one of our already operational virtual machines, start the migration process, and drive home in the meantime.

Once this was done, MrBlonde could be booted again and all MC servers were once more operational.

In the next part(s) we’ll talk about why we choose VMware, besides the segregation of the gaming platforms, and we’ll introduce you to all the new VM’s, there features and functions.

Part two: The datacenter trips (1 of 2)

We’ve split the upgrade process in 2 parts. The first time we went to the datacenter we installed a temporary 80GB Intel X25 SSD drive in MrWhite and installed VMware ESXi 5.1 on it. The previous night we had downloaded a copy of the virtual machines MrWhite and MrGreen to make sure if anything went wrong we had a backup ready to be restored.

The first challenge was figuring out if a SATA drive would fit on a SAS interface. To answer that question, apparently it can. :)

After fitting the SSD drive in the server and installing VMware ESXi 5.1 it was time to import the current virtual machines from VMware 4. This required a “virtual hardware upgrade” of the VM’s. This was a quick and painless process and worked flawlessly (yay for VMware!). Did a quick boot of the VM’s and everything checked out working just fine.

The second part of the first trip was installing the extra memory in MrWhite. We took 8GB memory with us, but after installing the memory in the server, it turned out that 2GB no longer worked and had to be removed from the system, hence the current odd number of 22GB memory. The machine has a total of 16 RAM slots to there is still plenty of space to put extra memory in if required.

Now we had created some extra harddrive space and tested out VMware 5.1 we could move forward with the upgrades. The extra HDD space was required so we could build up MrBlonde as virtual machine to keep the downtime for Minecraft as short as possible.

As a new Debian version was just released we went ahead and installed Debian 7 (Wheezy) in a new virtual machine, called MrBlonde as well (confusing, isn’t it? :) ). Mbl then went ahead to reinstall and move all Minecraft servers over to this new VM. We directly took the opportunity to move the MySQL databases used by Minecraft to this new VM. This way whenever we perform maintenance on MrWhite the MC servers no longer required to be shut down.

While mbl was building the new environment, we waited for our shiny new SSD to arrive. In the next part we’ll talk about the adventures we encountered installing that in MrWhite and the problems we had when reinstalling MrBlonde.

Part one: Rebuilding the servers

The first part we’ve looked at to make our community future-proof was our server hardware and its setup. Previously we had 2 machines, MrWhite, running VMware 4 and MrBlonde running Linux Debian 6. On MrWhite were the public TF2 servers located, as well as all non-gaming services such as the websites, stats, forums, mumble, IRC bouncers, mail environment, etc. On MrBlonde the other gameservers were located as well as the backup’s from MrWhite (websites, MySQL databases, etc.)

At SpecialAttack we try to setup stuff in such a way that we can give people access to the servers to run their gameservers from, as it’s impossible for one person to run it all. We try to supply the platform, and the community tries to supply the rest. Due to the way things were setup it was quite difficult to grant access to the systems without directly granting access to other services they had no involvement in, with the risk of them breaking it. Partially we tried to solve this with building chroot environments etc, but that wasn’t really a good solution. Another result was that we really had to fully trust people to not touch sections they shouldn’t. This made it harder for us to grant people access who had good intentions, but we didn’t know all that well (after all, most of us only know each other through our online activities, and not personally).

So, it was time to change this setup. We’ve had some real good experiences with VMware. In time VMware also improved greatly so it no longer had a negative impact on the gameservers performances running in a virtual machine. It was time to do some upgrading and re-installing!

As MrBlonde is fairly new, that machine didn’t require any hardware upgrades. It’s a powerful machine with loads of resources available. MrWhite on the other hand started to become dated, so could do with some new toys.

We’ve chosen to upgrade the machine with extra memory and a brand new SSD drive to speed up website loading times and improve database performances. The memory we still had left over from the old MrBlonde, so that could be re-used. MrWhite now has 22GB memory available for its virtual machine. For the SSD we’ve picked a Corsair Neutron GTX 240GB, the fastest affordable solid state disk we could find out there.

In the next news item we’ll take you through our upgrade process and the issues we’ve encountered and solved while rebuilding the machines.

It’s been quiet around here!

As you can see from the previous news item it’s postdate, it’s been a while since we’ve reported anything to you. This however doesn’t mean we’ve been sitting on our hands.

Admittedly we haven’t been doing things all the time, but we have most certainly been working hard the last couple of weeks.

As some of you might have noticed, we’ve kind of forgot to celebrate our 7th birthday this year, which was on the 28th of April. A month late, but nevertheless, happy birthday to us! (yay!)

In the upcoming week we’ll be posting a series of news items to inform the community on the changes we have made to prepare ourselves for 7 more years to come.

Stay tuned and check back on our website to read all about it!

Weekly community update

We know this is suppose to be a *weekly* update, but sometimes you just have nothing to report, so we missed out a few weeks!

In the meantime 2013 has arrived and we're slowely starting again at fixing and improving stuff around the community. Holidays are over, new jobs are started, so we're all set and ready again to provide you with another year of SpA gaming experiences!

Nowfor the news:

Community:
We've recruited a bunch new members! In no particular order these are;

  • Pizzaman194
  • Macavity
  • Haladir
  • sniper0zero1
  • bubbajim3

We also welcome back 2 members who have been gone for a while;

_9mmNL_ and Dunken, welcome back!

Starting this month we have a new TF2 manager and general recruiter, [SpA]JuncoPartner! You may all now suck up to him ;)

General:

  • Cleaned up leftovers from CS:GO

Killing floor:

  • Fixed Killingfloor servers not properly updating

L4D2:

  • Updated server

TF2:

  • Fixed issue with medic ubercharge plugin and the MvM servers (thanks hako for reporting)
  • Removed test CVAR that made the server cycle through maps
  • Fixed pl_goldrush escape for quickplay traffic
  • Removed servers "warserver, nocritz and all Halloween servers
  • Removed all advertisements
  • Updated gScramble plugin

TF2 Funserver:

  • Removed password
  • Updated (1.4.0) & fixed Saxton hale mod
  • Fixed mod's not working
  • Added proper mapcycle and settings
  • Installed Wario ware mod Wario ware-mod enabled maps
  • Added new maps for Saxton Hale mod;

vsh_2fortdesk_v8
vsh_courtyard
vsh_farm_feud
vsh_manncohq_v14
vsh_aperture_b3

  • Fixed a bunch incorrect settings in the Dodgeball plugin
  • Added new funmode "Freezetag", this is enabled on the maps cp_gravelpit, cp_granary and cp_badlands

KaM:

  • Added an extra server to test development releases on. The main server is now a "stable" server.
Syndicate content