Checksum Arcanius

When upgrading a couple of my servers to the latest Ubuntu, I ran into an issue with my php-based websites, including this one, where I was getting a simple yet inscrutible “File not found.” message instead of the php website.

Digging in, I found a few clues initially:

• The latest LTS of ubuntu (only a year old now :-p) had upgraded to PHP 8.1 from PHP 7.4, changing the php-fpm system I use to run these sites in a few ways
• Static pages still loaded fine, so nginx was basically working
• Eventually, by enabling php-fpm access logging and using this helpful answer, I verified that the issue had something to do with nginx no longer correctly passing the url to php-fpm.

Poking around a bit more and diffing the default 7.4 and 8.1 fpm server files, I found that it turns out that php-fpm 8.1 moved a bunch of config settings that make php work easily into snippets/fastcgi-php.conf. By including this file and removing the fastcgi_index and include fastcgi_params directives, I was able to restore order to my hosted sites.

Not exactly how I wanted to spend Friday morning, but onward! Hopefully this helps someone out there some day.

I finally bit the bullet and used lets encrypt to get certificates for several of the domains hosted by this server.

It was relatively straightforward in the end — though the program didn’t know how to completely automate the setup I had, the instructions were pretty good and the program was pretty clear in what it was doing.

I consulted a post on the nginx blog for guidance as well.

Say hello to https://arcanius.silverfir.net!

I’ve just finished upgrading the server behind this site to the latest Ubuntu LTS release, 20.04 “Focal Fossa”.

I think I’ve ironed out all the issues, but if you see anything weird, give me a shout in the comments or via other means.

I’ve been poking around the server hosting this site for the last few days and just took the time to upgrade to the latest LTS release of Ubuntu (18.04). Things went smoothly, I think — the only manual intervention required was moving php-fpm from 7.0 to 7.2, which involved only a straightforward copy of the old config files.

Just before the upgrade, I moved to a model of a separate php-fpm instance for each of the sites hosted on this server. Each one is running as it’s own user, so in theory the security guarantees should be much stronger (at most, a single site might be compromised — there should be no way to damage a second site if a first site has a vulnerability).

As always, if you notice anything amiss, please let me know.

A few days ago, I was poking about “micro”, the AWS EC2 server behind this site. I noticed that a new LTS release of Ubuntu was out so I decided to upgrade.

Unfortunately, I got distracted amid the upgrade and forgot about it, and then it took my brother to inform me that things had gone awry: every page load was returning “502/Bad Gateway”. So, just like last time, I had to dig in to figure out what was going wrong. I started looking into it and the problem ended up being multi-faceted.

First, I needed to finish off the install which I had rudely interrupted by rebooting the server while the do-release-upgrade was stuck at a prompt. Whoops! Fortunately, apt-get is nice enough these days to tell you the invocation you need to resume the upgrade — something like dpkg -a. Regardless, I got that resumed and finished up, then I set about seeing if everything was working.

Well, of course it wasn’t! First of all, nginx was set to work with php5-fpm, but this new release ships with php7, which has been put under the more generic name php (which seems like a good move, even if it’s backwards incompatible, because it allows for compatibility going forward, whereas the old method did not).

The first step was to update the nginx configs so that it would talk to the right unix domain socket to communicate with the upgraded php-fpm package. Once that was done, however, the pages started showing up blank, but with HTTP/200 responses, as if everything was working fine. Some searching led me to the regular place where all sysadmin questions go to be answered: serverfault.

Sure enough, this was exactly the issue I had, and adding that line to my nginx config made the websites start loading again.

Now, I just have to figure out the email situation again…

A couple years back, I moved from a full tower for my file server to a small HP Proliant MicroServer (I’ve been very happy with this move). When I made that move, I went from 4x750GB in a RAID 10 configuration to only 2x750GB in a RAID 1 configuration. However, I keep taking photos and I noticed I was running out of space. Since I had the extra Hard Drives just sitting around, I decided to put them back into use. the hardest part was that there are only four HDD slots in the server, and I was using one of them for the boot drive.

However, I had long noticed that there was a 5.25 inch drive bay in the server that was conspicuously unused. All I needed was an adapter. It took me two years to finally get around to this project, but today was the day! I found a drive bay adapater and SATA power and data cables, then got to work. From a physical perspective, it was fairly straightforward once I had all the parts; the hardest part ended up being finding the extra SATA port on the motherboard.

I was a bit wary when I booted the thing — the hard drives I used had previously been a part of the 1.5TB array that the other two came from. Fortunately, everything went smoothly. It was then a rather simple matter of finding this excellent guide on ServerFault to rebuild the array into RAID 10 once again.

Yay technology; internet FTW!

Based on an answer on serverfault, but modified to work on linux:

Use this one-liner to get the bash and sshd processes currently hanging onto the old screen.

ps aux | grep $(ps aux | grep [s]creen | grep -o 'pts[^ ]*') | grep 'sshd\|bash'

Kill these processes and screen is recoverable before the long and arbitrary timeout.