Joel Strouts
About Articles & Projects Tutoring

How Web Servers Work and How to Set Up Your Own.




Summary


I wanted to learn PHP (a programming language). It turns out PHP runs on a server. I didn't have one, and there are ways of spoofing the functionality but I was interested in doing it the hard way (so I'd learn better). I didn't know much going in so I learned a lot. Since I'm sure I'll soon forget everything I'm going to write it down.


There are lots of good resources already, but starting from the bottom I found I was missing a lot of context that was taken for granted. That makes this a sort of guide to guides - an attempt to lay out the key concepts in plain english without getting into the details. Then hopefully you'll be on the same page as the more techy explanations. (though I do have some troubleshooting tips from my own limited experience at the end).


Sections:


* How Servers Work

* Loading a Website: Down The Pipes

* Setting One Up Part 1: Server Config

* Setting One Up Part 2: Network Config





How Servers Work


I thought I already knew about servers, they were those big blinking boxes with lots of wires going in and out of them that I assosciated with google and 'cloud storage'. Something to do with networks. What did they actually do? not so sure. Turns out it's in the name! They serve things. Web servers serve webpages, file servers serve files. Some notable types of servers:


* Web Servers
* File Servers
* Print Servers
* Mail Servers


I am only really writing about web servers, though file servers come in handy so they are mentioned at the end. If I learn about mail servers I'll write that up too but I don't know enough about them yet.


The functions of these servers differs but in each case the physical thing being called a server is just a computer. Those industrial blinking boxes - they're just computers that were designed with being a server in mind. A general purpose computer can do the same things even if that's not what it was optimised for. Print servers used to be a seperate piece of hardware that you set up alongside your printer. Now it's a little bit of computer integrated into the same case. Still just a computer.


A computer designed to be a server could have different features like redundant power supplies so if one fails it keeps running. The big difference is that general purpose computers weren't designed with 24/7 runtime in mind.


To make your computer act like a server then, it's just a case of installing the right software. At the moment it's running some not-server software but you could wipe the disk and start it back up with server software and it would chug along all the same.


Like with hardware, the software that tells the computer how to be a server differs in seriousness. You can run server software as just another program on your computer - plex is a program you can run that lets you play music and film through your TV. Your laptop acts as a media server. This means if you close your laptop while the film is playing, the connection is lost and the film stops. This is why you might want to dedicate a machine to being a server all the time. The thing is, by committing a machine to life as a server you lose the need for all the usual computer things like microsoft office and solitaire. If all a computer is going to do is serve webpages, then that's all it needs to understand. Having it know anything else is just a waste of computer brain that could be used for serving web pages better. Server operating systems do just this. Features are stripped away until you're left with a very barebones, very good at one thing computer. Features are also stripped for security reasons; a good server is a reliable server, and the less features there are, the less can go wrong.


The server this website is hosted on runs the ubuntu server OS, which is so stripped back it doesn't have a mouse or clicky buttons or pretty pictures. It just displays a black screen with text on.


That's what servers are, now to understand how web servers do their job.




Loading a Website: Down the Pipes


Recap: A server is just a computer doing a particular job like serving web pages. Now to talk about what serving web pages actually involves.


The thing you use to browse the web is your web browser. It's chrome, or safari, or internet explorer, or something else. It has an address bar at the top or maybe at the bottom on a phone's browser. When you type "joelstrouts.com" into that address bar and hit return, 300ms later (lets just say 300) you are looking at this website. I am going to talk about what happened in that 300ms.


First is the request, you essentially said "there is a webserver somewhere going by the name of 'joelstrouts.com' and I want to have a look at its contents!". The browser can't grab any files just yet though because joelstrouts.com is just a nickname, not a real address. As far as the computer is concerned, servers are identified by strings of numbers and dots like '127.0.0.1'. It needs some way of translating the nickname to an address it understands, a phonebook of sorts. This comes in the form of a 'Domain Name Server'. They are another sort of server that recieve nicknames and respond with their proper addresses. Google has DNSs, and your internet service provider probably has their own DNS too. The first thing your browser does is ask a DNS (whichever one it's set up to use) "what's joelstrouts.com's address?" and the domain name server responds accordingly.


Now your browser has an address it can make sense of, it sends a new request to that address. The request is for a webpage, but not a specific one, just whatever the default is. The server is set up to serve a default page for just this scenario. It is a file on the server probably called "index.html" or "index.php". The server packs up the appropriate file and sends it back to your browser. When the browser reads the file it may find that it asks for a couple more files too, like one for a pretty font, or for a couple of images, in which case it fetches those too - this time without need for the DNS lookup.


This is a slight simplification though. That address the DNS responded with just points to a "router". A BT home hub, in this specific case. You were looking for a server and you got a router. The server is actually just another device that is connected to the internet through that router. This is like sending mail to a hotel without specifying which room number it's for. Of course, it worked, you are reading this so there must be more information passed along somewhere or the request wouldn't find the right place. Where is the information though? That DNS lookup for joelstrouts.com really does point to the router and nothing more. The answer is that string of characters before the address. Sure, you may have just typed in joelstro... but when you hit enter your browser will automatically stick a "http//:" on the front of it. That is how the server at the end knows what you're after - in this case a webpage because "http" stands for hyper text transfer protocol. It is helpful here to know that the heart of a sites is contained in "hyper text markup language" files, so 'HTTP' is the way of transfer them.


Just like routers, the different devices connected to your router ("on your private network") are identified by strings of numbers with dots intbetween, like maybe "192.168.1.254". When you connect your device to the internet it gets assigned one of these 'addresses' by your router. The router is like a hotel and every device that checks in is assigned a unique room number. 'Ports' are how the router gets messages to the right rooms. There are internal ports and external ports. If everything is chained together right these ports will funnel messages where they need to go. The request comes with a port number (HTTP requests are to port 80. The file transfer protocol uses port 21), you can link external ports to internal ones, and internal ports can be linked to the addresses of your devices. I have configured my BT Home Hub to link external port 80 -> internal port 80 and internal port 80 -> my old laptop!


Your router will say "I have recieved a request on port 80, and I've been told to pass those along to the internal port 80, which points to Joel's server - so that's what I'll do". That's the whole route to find the right server and once it's found the job's a goodun. As long as the server at the end of the chain is set up to deal with the request, it will know where to send the files back to. Once it sends them, your browser will load them and you'll see the website!


So that is the order of operations. Now you know what should happen if there are no hiccups. That means you'll do a better job of figuring out which link in the chain's broken when one surely does.




How To Set One Up Part 1: server config


If each of these steps is working properly your server will be online.


1. You need web server software running on some computer


There are many options here, you can run a server inside a virtual machine on your general purpose computer, or use a disk to upload a web server operating system onto a dedicated machine. If the website will be seen by other people it makes sense to use a dedicated machine so you can have it running all the time. Ubuntu server OS is a solid choice, because it is free. It does have a learning curve all of its own though. Do some research here to decide what suits your needs best, search "different web server software" and once you've chosen what to use search "how to install [your choice]".


2. You need your server to be connected to your router


This is straightforward enough. I will say an ethernet connection is a safer bet than wifi. I had some real headaches trying to connect my ubuntu server to the wifi, and reinstalling the OS with the ethernet cable plugged in worked first time. Troubleshooting for this step will involve commands like "ipconfig". Search "[your server software] not connected to internet". You may find yourself trying to make sense of IP addresses at this step, and it could be helpful to know that 127.0.0.1 is the loopback address. That is to say, if your server only reports that it is connected to 127.0.0.1 then it is not connected to anything at all. That is the localhost. You can tell other things from IPs too - private IPs and public IPs are in different "ranges" so knowing these ranges will help you make sense of the diagnostics you might find yourself looking at. Private IPs are the addresses of the devices on your home network remember, and public IPs are for routers like the one you connect to the internet with. Private IPs (internal addresses) look like: 192.168.blah.blah or 172.16-31.blah.blah or 10.blah.blah.blah. For more information search "IP ranges"


You may also be required to enter other information like a "DNS" and a "Subnet mask". Domain name servers were explained earlier, your server needs you to assign it one so it can look up stuff. Many people just use google's DNSs at 8.8.8.8 or 8.8.4.4. A subnet mask is just a fancy term for how your device knows which addresses are in the private network and which ones are not. This will depend on what range of IP addresses your router dynamically assigns to the devices on your private network. There is a great brief explanation of this on youtube called "Rapid Explanation of Subnet Masks". It will look something like 255.255.255.0.


3. You need your server to have it's port 80 'open'


This is a firewall thing. How you make sure this is the case depends on your server software. On ubuntu server OS there is an 'uncomplicated firewall' and telling it to open port 80 is as simple as typing the command 'ufw allow 80'. To figure out how to do this on your server search "open port 80 [your server software]"


4. You need a file on the server that contains your webpage (and it needs to be in the place your server expects to find one)


AGAIN, this depends on your server software. The server expects to find a file, most likely an index.html file (by default) in what is called the "web root". This will just be a folder in the server file structure somewhere. You can find out where yours is by searching "Where is web root on [your server software]". If you are paying for a server hosted by someone else you'll have to check with them how they've configured it. You can change the default file your server will respond with (perhaps to a php file) in the server configuration. Search "how to change index.html to index.php [your server software]"




How To Set One Up Part 2: Network Config


1. (If you have a domain name) You need the domain name to be linked to the right IP address (the one where the server is)


If you don't have a domain name (getting one is just a case of coughing up the money) then you'll just have to use the IP address directly. This will be the public IP of your router with a http//: infront or a :80 on the end like blah.blah.blah.blah:80. Assuming you do have one though, linking the domain to the right IP will be something you have to configure through your domain name provider. Search "[your domain name provider] how to point domain at IP". It will involve adding an "A record".


The IP address of your server is the Public IP of the router through which it connects to the internet. If you're connected to the same network you can find out what that is by just googling "IP address". If you pay for a server then it's IP address will be something your the provider of that service will have to tell you.


2. You need the router to take requests on port 80 to the correct place


This is called port forwarding. This is done in the configuration panel of your internet router. The place where you find this depends on who is providing your internet. For me, on BT I can access all these settings by typing "192.168.1.254" into the address bar of my browser whilst connected to the router. To find out how to set up port forwarding on your router search "[your internet provider] how to set up port forwarding". This may get a bit technical but if you remember the key steps you're trying to set up then you should be able to make sense of it. You want the external port 80 linked to the internal port 80, and the internal port 80 linked to the private IP of your server. Simple as that!


Your router may give out a different address every time something connects to it, which can be a problem for port-forwarding because if your server gets a new address the router may now be pointing requests in the wrong place. You can most likely get around this with some more sophisticated port forwarding where the internal port points at the name of your device rather than a fixed address. This all depends on your router.


If all of that is set up okay, it will work!




Notes


there are two things that have made the experience of interacting with my server much more convenient: SSH and setting up and file server.


I said I would mention file servers but this is the first mention of SSH so let me explain what that is. It stands for secure shell. "shell" is a word for the command prompt of the computer. On your general purpose computer this is a program you can open and type commands into. This is the primary way that you will interact with your server. This is the place where you will type in commands like "ufw allow 80". To access this command line you will of course have to have access to the computer that is running your server. What if you don't want to have to be in the same room as the server to give it commands though? It would be better if you could just leave it in a cupboard and operate it from another laptop. That is what SSH is for. By establishing a SSH connection between your computer and your server you will be able to send commands to the server through the shell on your laptop. Your server will have user profiles with passwords - connecting to it by SSH is as simple as typing "ssh [user]@[private ip address of your server]". It's very convenient!


A file server comes in handy for a similar reason. Once you have set up your SSH 'tunneling' you will be able to edit files and make new webpages from the command line on your personal computer. That is better than having to sit in the cupboard to do it, but editing files on the command line is not something many people like to do. By configuring your server to act as a file server too, you will be able to view the files on the server with your other computer just as if they were really there. You can open them with any program you like. The best way to do this depends on your particular setup but if you got this far you'll definitely be able to figure it out. For ubuntu server you can use "apt-get" to install a 'samba' file server.




I hope this helped, or at least got you far enough that you could make sense of other, more technical articles!





Check out a project of mine:


Factor Visualiser Project

An app that factors your input and displays the result on an interactive grid. Made with vanilla html, css & javascript.


How web servers work, and how to set up your own.

The guide I wish existed when I set up my own webserver.


About


This is the personal website of Joel Strouts. I use it to share projects I am working on and to learn about web development.