Skills you need to manage a (Linux) web server

Curious to manage your own web server?

• What skills do you need?
• How long will it take?
• Is it even worth it?

All the answers to deepest darkest server management desires are buried (not-so) deeply inside this post!

I promise to make it easy. And make you feel comfortable. Nothing I say below will scare you away or make you think that you’ll never ever be good enough. Truth be told, I never wanted to be a server admin. It was never my fantasy and I hated it. But I was forced to learn it because I couldn’t trust my businesses to other people who were never around when I needed them.

Knowledge is power…sure…but finding out you’re so much more capable than you ever thought you could be…that’s life-changing.

…are you ready for the red pill?

1. Linux commands

This is IMO, probably the hardest technical part of web-server administration for beginners. It’s because you don’t know how to do basic shit. You don’t even know what to call the things that you want to do…and therefore can’t look them up. Then when you do finally find the commands online, they don’t work!

There are 2 main “versions” of Linux distributions (RHEL/CentOS & Debian/Ubuntu)

Yes, there’s hundreds of different Linux distributions out there but they’re all basically a fork of either RHEL (Red Hat Enterprise Linux) or Debian. And the main workflow difference between them is that they 1) have different package managers and 2) store certain files in different places.

• RHEL-based distributions use yum to manage packages. Debian-based distros use apt.
• RHEL stores website directory in /home/user/public_html. Debian uses /var/www/html.
• Keep in mind: most Linux web servers nowadays will be on CentOS or Ubuntu. Sure, you can use other Linux distros but most of today’s web server guides are designed for CentOS or Ubuntu.

The explanation above should clear up most of your confusion between different Linux distros. If you see guides showing yum and /home/user/public_html, you’ll know that’s probably for CentOS or other RHEL-based Linux. And if you see apt commands or references to /var/www/html, you’ll know that’s probably Ubuntu or other Debian-based Linux.

There are still other everyday differences such as their default software. (For example: CentOS default firewall is firewalld and Ubuntu uses ufw.) They also have their common server logs and configuration files located in different places. Just think of it as cars from different manufacturers. They all have the same parts but put in different places and accessed or managed in a different way.

Why aren’t other Linux distributions popular for web servers?

• That’s another question for another time!
• CentOS and Ubuntu are the main ones today and that’s all you need to know for now!

“What Linux commands do I have to learn first?”

• Server Shell Commands CHEATSHEET (for newb admins)

Try my guide above. Learn how to see basic server info, moving around directories. Moving files and folders. Copying, deleting stuff, changing permissions. Using wildcard commands. Then how to install packages. How to create, read, and edit text files. How to connect to other Linux servers.

Play around and see how far you get. Whatever you can’t do…I suggest hiring somebody to show you within 5 minutes rather than wasting many hours filling your head with all the wrong ways of doing things. Trust me on this; there’s so much shit to learn, you don’t have time to be wasting it on basic stuff.

2. Setting up web servers

These are web servers that you set up to LEARN.

If you’re building a server to learn from. There are 2 routes:

1. Use one of those simple free stackscripts. EasyEngine, Webinoly, etc.
2. Set up from scratch using one of those full server setup guides.

I prefer method #2 for learning. It’s far more powerful as you really start to understand what each service/package is used for. How it sets up, common commands to configure it, where to go when it doesn’t work right. #1 is good if you want something that works easily without much effort but then where do you learn and where do you get to practice configuring things?

You’ll probably have lots of anxiety at this point. Because one guide says to do “abc” and another says do “abc -f”, and another says “don’t do abc, do efg”. And then when you try to learn the difference about each one, it takes you further down a rabbit hole about something you never heard of in the first place!

You’ll often find that many commands simply don’t work exactly as in the guides. This can be for a number of reasons. But the most common beginner issues are listed below:

• You’re on the wrong distribution (following a Ubuntu guide while on CentOS server, or vice versa).
• Your VPS or server company has pre-configured firewalls that need to have ports opened.
• You’re using the wrong kernel version (like following a guide for CentOS 6 while using CentOS 8).
• The commands you want to use need another package installed first. (And quite often, that package you need to install depends on other packages as well.)
• The commands you want to use are for a different version of the packages you’re using.

So basically, almost all of your setup problems are due to wrong Linux version, or need to open firewalls, or you need to install/re-install some packages.

And now for the web servers you set up to USE.

Servers that you actually plan to use for live sites are called PRODUCTION servers. In the development world, this term is used to label servers that actually serve live websites to real users. These are obviously more critical in management than a development server or test server.

Don’t mess around here. Set everything up neatly. Set up some security. Set up some monitoring. A GUI-based control panel is also important for end users. When in doubt, pick the more conservative option that doesn’t require so much manual configuration and hacking. That only leaves so many things for you to account for later. What if you want to hire somebody else to fix your server…ahh, you gotta remember that you changed this port, or disabled this service, or moved something to another place, etc.

• Be careful when you follow guides that over-customize things!

3. Common web server tasks

See how easily you can do these day-to-day tasks. Would be great if you practice them on your own to be comfortable before you need to do them. If you’re using a GUI-based control panel…it’s still helpful to learn both ways (GUI method for teaching end clients, and CLI method for faster management yourself and also when the panel doesn’t work.)

• Create and delete users.
• Changing file/directory permissions and ownership.
• Moving files/directories.
• Archiving and extracting TAR.GZ files.
• Create FTP users.
• Manage databases with phpMyAdmin.
• Database management (create, copy, export, import).
• Changing PHP configurations.
• Starting, stopping, restarting key web services (web server, mysql, ftp, ssh, mail, dns).
• Firewall management – open and close ports. Whitelist or blacklist IP’s.
• SSL – generate, delete, copy, import SSL certifications.

4. Server configuration

No such thing as “best server configurations”.

People try to ask me all the time…“Johnny, what are your recommended server configurations”. And it makes me laugh because that’s not how you should do it. How many hot dogs and bottles of wine should you buy for your next picnic? It depends!

If you don’t know the “optimal” server configuration, please stick with the defaults. Yes, they won’t be the best performance but you will learn. Some things will slow down and bottleneck and you’ll adjust them. Once you know what to adjust, you can try different settings…go too high, or too low and see how your server responds. Simple as that!

This is something you can really only learn with experience. After a couple years of hosting various websites…oh yeah, you’ll get a feel real quick of what each site needs or what each server needs. It’s like owning different pets. You get a sense of when they need food, water, whether a certain behavior is normal or means something’s wrong.

Oh yeah, it helps to work side-by-side with an experienced admin. Preferably one with lots of high-traffic experience and not just a Linux script-kiddie who copy-pastes the same thing for every client. Preferably also not a fancy server buyer who prescribes an expensive AWS solution for every client.

This IS such a thing as “best workflow”.

Ok, I somewhat lied. I do have some “favorite server configurations” but they fit my workflow and style of configuring for client sites. Some admins like to optimize a certain way, using more memory to boost speed but others prefer to save memory and optimize by closing connections sooner. Some admins like absolute lockdown security, others like it more open (not to annoy real visitors) and then manually ban as needed.

It really depends on your style of server management. Is this YOUR server that you’ll be in everyday? Or for a client that you want to be more hands-off? As with anything…overly aggressive or “strict” settings will require more manual tuning over time.

5. Resolving issues

Check the server logs

As cryptic as Linux can be sometimes, I continue to be amazed at how easily most problems can be found by checking the logs. Something doesn’t work? Just check the logs! It’ll show you what happened. Then go research the error or status that you see and voila, issue is solved!

Now for the not-so-easy part of server logs.

• You have to know which logs to check when issues arise.
• You have to enable the logs. Sometimes, the info you need isn’t logged by default. And/or maybe you need to install another package to monitor it.

Basically, the hard part about logging is knowing WHAT to log and WHAT to check within that log. But if you already knew what to look for…then you probably already know what the issue is. It’s a funny catch-22.

“Which logs do I have to know about?”

• The default logs are the ones for all your services (Apache, mysql, ftp, ssh, firewall, etc).
• Then there’s monitor logs are the ones recorded by any monitoring software. Like atop. They monitor your server resources and the processes using them.
• Just about every service has a log. If something won’t start or has errors, you can always see why in the log.

Look up the services you’re using and where their logs are located. In fact, I recommend you go through each log right now and get a feel of what kind of information is in them. What kind of status or errors they record. Then when things happen later, you already have a good sense of where to look.

The importance of manual logs

For most new servers, the problems happen on their own and usually unrelated to the server. Usually just some service or port that need to be allowed. But for an older server and with many users on it already, the problems can be because of server changes YOU made. Maybe you updated or reconfigured something and now some sites or services stopped functioning.

For this reason, it’s important to log everything you do.

• Write it down somewhere so you can revert if you find out about a problem one week later.
• Too lazy to log? It’ll cost you that much more time re-provisioning a new server from scratch!
• The only time I think you don’t have to log everything (but should still log some) is if you’re in there every week and know the server like the back of your hand.

6. Have a good admin available

Can I really manage a web server all by myself?

Because let’s be honest, that’s the real question. The answer is “yes” IF you have someone supporting you. I think owning a web server is like owning a car. Anybody can do it as long as you have people around to help you for things beyond your technical ability.

Is it more frustrating than a basic webhosting plan? Yes, totally. But your experience can vary and depending on how it’s setup, you might not have any pains at all. After all was said and done, I loved my web server management experience but will definitely say it’s not for everyone. Just like how some people prefer working for a company rather than running their own company.

Find several good sys-admins

This isn’t the place to go over what makes an admin good or bad. You’ll have to try several to know who fits you best. Just some thoughts below…

• Some admins are really good and can fix any problem but hate explaining what they did.
• Some admins are overly-controlling. Insist on making changes to 100 places, but good luck figuring out what they did if you ever break up.
• Some admins do only what you ask for…and never what they personally recommend. If you don’t specify, they don’t do!
• Some admins like to default to expensive “paid” solutions all the time (for their convenience but at your cost).
• You should always have at least 2 or 3 admins on hand, especially to cover all 24 hours of the clock!
• Some admins will take the job even if they don’t prefer your setup. It’s better to find admins who work in your stack daily. (Apache vs LiteSpeed vs NGINX.) Or at least, you should let them rebuild your server in a way that’s most comfortable for them to manage.

Put your ego aside

There have been countless times in my server management experience (and still happens today!!!) where I can’t figure out an issue and have no choice but to come crying to my senior admins. What I can’t do in 5 hours, they do in 5 minutes. And of course, the answer is always something I almost figured out had I dug a little more.

Then I ask…“OMG! YOU’RE SO AMAZING! WHAT DID YOU DO?!”…

…and then I learn.

I try not to push the bailout button as much as possible, but inevitably I have to. You have to put your ego aside when you have clients trusting you with their financial livelihood. It’s not just their web server, but their business, their baby. They deserve the best and if that ain’t you that day, you need to step aside for the real professionals.