“End Titles” by Danny Elfman, Nick Ingram from the major motion picture Big Fish 🎵
A few months ago I attempted to upgrade to WordPress 6.3…
It backfired: horribly.
It landed at 4 different Lightsail environments.
I decided to revert and give up for a bit, while simultaneously beefing up the compute and storage power of this website. I’ve mentioned before I run this website lean but I still do backups and upgrade. I decided I’d try again after some fixes had gone through by plugin providers. There’s still issues, but I upgraded anyway.
How I Upgrade WordPress
When I do a WordPress core upgrade in Amazon Lightsail, I take a snapshot (in addition to recurring snapshots) and then create a new instance based on that snapshot.
I keep the original site running and upgrade on the new snapshot. If I run into issues, it only takes a Static IP swap for me to go back to the old site. My simplicity is because (1) this is a personal website and (2) there is a 15 second revert backup plan if it backfires and (3) like 30 people read this site. I know who they are – they come for the puzzles and jokes I hide in other posts, watching me epically fail at life, and not for 11 9s of uptime.
Also it’s cheap.
I then stop the backup but keep it around for a month. This means I only have 4 blog posts to carry over if for some reason I absolutely need to go back to the older version. When things have been okay for a month on core or an upgrade, I then delete all old snapshots and the old host. The reason 6.3 wasn’t working well for me is when I was typing actual blogs, like I am now as a test, the more I wrote…the slower in real time my site got.
It would be like playing a platformer but every time you jumped, you weren’t able to jump as high until eventually jump stops working.
The more I look into it, the more I realize that the issue is in core itself or isolated to Chrome and that I am not familiar enough with post.php, ajax calls, why it would get stuck in pending, and publishing functions to contribute a fix. By the way, I’m still seeing this issue when using Google Chrome and it especially epically breaks when previewing blog posts – I can write for about 10 minutes in Chrome and then I’m done.
Currently, I’m typing in Safari: Where it is fine???
Fact vs Fiction in Bugs: Updating Themes and Plugins
This brings me back to Linux. When I upgraded this time, I also had to do theme updates (I had already dealt with plugin update problems). However…in 6.3…there have been changes…to how both themes and plugins…are…updated. Which meant this happened:
My themes wouldn’t update because they couldn’t mkdir (make a directory) for a magical directory I’ve never heard I needed before. Turns out! It’s a new feature of 6.3.
“When updating a plugin or theme, the old version of the plugin or theme is moved to a wp-content/upgrade-temp-backup/plugins/PLUGINNAME or wp-content/upgrade-temp-backup/themes/THEMENAME folder.”
Good idea. I like that idea.
I immediately put on my “of course it’s a Linux chmod problem” face and logged into AWS. You see, if “you” can’t make a directory (in this case wordpress can’t) it’s because in wp-content the user or group or daemon creating the directory doesn’t have permission to do so. And generally, you probably don’t want to give it these permissions all the time – only when you actually need to do the upgrade.
Let’s check that theory though and learn a little big more about Bitnami’s stack, which most of these WordPress Lightsail distributions use.
Clicking the terminal button in Lightsail will automatically open up an ssh connection using your existing AWS credentials from the console in a snazzy terminal. If you cd stack/wordpress and then ls you’ll see everything that’s there.
My first thought was – I need to know what the permissions are for wp-content and also see if the folder exists. Which is why I checked and realized: It doesn’t. You could create the directory yourself at this point by running mkdir but you still won’t be able to update the themes from WordPress itself if you do that. As I mentioned, it’s a permissions issue with Bitnami…something wants to do things with that folder.
The Big Fish: Linux Permissions & Symbolic Link Facades
I ran ls -l to list all the permissions for /stack/wp-content. I realized that I actually wanted to see the permissions for wp-content, not what was in it. This led me to realize…it’s a symlink.
Symbolic links (or symlinks) access permissions don’t actually matter because they point to another file. As this documentation points out “The symbolic notation, lrwxrwxrwx, is the only set of access permissions a symbolic link can have…In conclusion, symbolic links have irrelevant access permissions. Users are only prevented from operating on a symlink by the permissions of its parent directory and the target file.” This meant where I actually needed to be was in another castle.
I got on my Linux airplane and cd /bitnami/wordpress to find the permissions were, as expected based on the Bitnami documentation, owned by the bitnami system user and the daemon group with permissions 775. This means bitnami user (Aka me right now) can read, write, and execute. But wp-config.php, the problem maker, was 640 because it needs to be further locked down.
Pulling back a bit as a simple concept – if you, or code, doesn’t have permission to perform an action on a file or a directory, it can’t. wp-config.php controls a lot of aspects of WordPress — in a normal state where we aren’t making upgrades or updates, we generally want to keep it locked down. I’d like to acknowledge that if you are new to this – these numbers of 755 and 640 seem like a foreign language. Don’t fear, they just represent permission settings when running chmod (change mode) in numeric mode and they translate to those letters you see (rwx or read, write, execute as this blog is titled).
If you want to read more deeply on Linux permissions, this is an absolutely great walkthrough and I would explore any host you own to understand your own permissions in the stacks you have. When in doubt you can also use the Chmod calculator. For the line -rw-r—– the first set of permissions applies to the owner (in this case bitnami user) and the second to the group. This means in the wp-config.php permissions rw- means the owner (bitnami user) can read and write but not execute. Then, r-- (daemon) means the group can read but not write or execute, and then — which is “other users” (or public) not in the file or folder’s group – they can’t do anything. Hence 640 instead of 664 (which would give daemon more capabilities as rw- including write).
To fix this we need to make some changes. The group (daemon) can only read wp-config.php – best demonstrated in the following example 1 in this post: https://raddy.dev/blog/aws-lightsail-fix-wordpress-file-permissions/. As Raddy.dev explains – we need to update this in order to let our upgrades happen.
I ran each one of these one at a time:
sudo chown -R bitnami:daemon /bitnami/wordpress/
sudo find /bitnami/wordpress/ -type d -exec chmod 775 {} \;
sudo find /bitnami/wordpress/ -type f -exec chmod 664 {} \;
I then proceeded to upgrade the themes in WordPress itself.
Awesome. Then I locked wp-config.php back down with sudo chmod 640 /bitnami/wordpress/wp-config.php
You can see the full set of changes here:
You’ll notice that Raddy’s blog does leave out an important configuration to change back depending on your plugins. If you are using the W3 Total Cache plugin you may see the message “/opt/bitnami/wordpress/wp-content is write-able. When finished installing the plugin, change the permissions back to the default: chmod 755 /opt/bitnami/wordpress/wp-content. Permissions are currently 775.” While this deviates from the documentation…I agree with the idea and decided to set it back using:
chmod 755 /bitnami/wordpress/wp-content
Depending on how you like upgrades to happen – automatically or not – these settings are going to be what gets you in trouble.
I’m looking forward to my automatic plugin updates breaking in the near future and further locking this down now that I’ve done a post on my permissions and users – I think this warrants a followup about changing owners and creating users in Linux systems.
But mostly, I’m looking forward to figuring out someday why the metaphorical WordPress “jump” seems to only break in Chrome and not Safari.
Which, I guess, I’ll be using for a while because it now sucks to write in Chrome.
