First of all, I would like that say "Thank you!" to all the users of ChatterboxTown.US and to the the people at JMP.chat for your patience and kindness during the extended outage that occurred between July 19 and July 20. Basically, I woke up that morning on July 19 to find ChatterboxTown.US completely unreachable. I immediately contacted IONOS hosting company support and they did a quick look at my resources monitor and claimed that my resource usage (CPU and RAM) was maxed out and that was why my system was unresponsive. Now, for the last year, ChatterboxTown.US has grown to about 750 concurrent online users with resource usage still remaining at less than 25% capacity (even up until the night before). So this was a very surprising analysis to me, but I could not argue because I could see on my control panel that the system monitor was in fact showing what they were saying. They said I needed to upgrade my VPS to a higher capacity and that would solve the problem. I went ahead and agreed to upgrade the VPS to one that would cost me 5x more than what I was currently paying, but if that meant getting things operating again quickly it was the least of my concern at the moment. So, the upgraded VPS was fired up, but again the system was unresponsive upon boot up and resource usage was still showing max usage. At this point, I thought maybe my system OS had really bit the dust. My next option was to rebuild the services on a new VPS. I had them deploy a new VPS with the original capacity. Within minutes of getting the OS running and updating, the new VPS became unresponsive as well. Reporting back to IONOS support, they could see and agree that having two separate VPS lock up like that was bigger problem and they escalated the issue to their data center techs. Anyway, I had to go to work by that time, so I figured maybe I'll hear back sometime during the day. Halfway, through my work day, I called support again to asked when I should expect to get a reply. It was then that I was told the data center techs are in Europe, so they probably wouldn't even see the support request until their work day started (like 11pm my local time). Not much else I could do at this point but wait and start making a plan B.

So, when I woke up the next morning to still find no reply from tech support, I started shopping for another VPS host to start rebuild my service. It was about 29 hours since the outage (I did finally get an automated email saying the techs were investigating the problem), while I was working on preparing the new VPS at the new host, when I decided to check my IONOS VPS one more time. This time my VPS booted up and remained online. ChatterboxTown.US services were running normally again. It was an hour after that I received an email stating they "fixed the problem" without any further details.

Well, good news and bad news for me. Services were back online, but now on a VPS that would cost me 5x more than my original VPS, and which I really don't need since it was not the solution to the problem. I took this issue to IONOS support. How do I restore my contract back to what it was before I needlessly updated my VPS? The answers were rather unsatisfying. First, they could not downgrade the VPS to lower capacity tiers. Second, their data center techs could not (would not?) be able to take my virtual hard drive and just attach it to the appropriate VM. Third, the only way to downgrade back to my original tier was to manually rebuild my server myself on a new lower tier VPS (essentially going through the whole disaster recovery process when I really didn't need to). Fourth, even though I'm going back to my original VPS capacity, the prices have now gone up a few dollars. Of which they cannot (will not?) revert back to my original contract price. In a weird way, I feel like the landlord somehow orchestrated some kind of disaster to make me sign a new lease into a unit that is exactly the same as the one I was living in, only now rent has gone up.

Well, that's what I did. I rebuilt and restored my backups on a new VPS (of which is the same original capacity as I've had, with a couple dollars more on the monthly bill). I was given a $25 voucher for future invoices.

The VPS on the other host? Yes, I'm keeping that too. It's already on standby in case the IONOS VPS needs 24 hours to reboot their hypervisor.