Hosting a website and setting up The Pi

I thought, "I should get a raspberry pi so I can do techy projects with it, that'd be fun and would show off my skills I think"
I ordered it via Canakit
I found a cheap monitor, keyboard, mouse on Kijiji

Monday

I bothered a techy friend of mine who knows a lot about web development to help me drive around to pick them up
I served her tea and asked them a ton of questions about how hosting your own website works (I wrote down the questions ahead of time)
I had previously picked up some basic HTML/CSS for a different project, so I sketched out a potential website layout and structure to use for this project
my friend gave me lots of details and I wrote down a general overview of the steps, I didn't remember most of the details but I knew the key words I had to google like "port forwarding" and "cloudflare domain registration" and "caddy docs".

Notes from that conversation

How To Self-host a Website:

construct HTML/CSS files
- the standard place on Linux to put them is in /var/www/myfile (what if I'm hosting multiple sites from here? not sure how that works. Multiple directories within /var/www?)
install "caddy"
- launch at boot with systemctl, possibly run "sudo systemctl enable --now caddy.service" (look up details of what command does)
configure caddy
- goto /etc/caddy/caddyfile
- look up the caddy docs (google "caddy docs") to figure out how this config file works
find the Pi's IP
- google "what is my IP"
- run "curl icanhazip.com"
do "Port Forwarding" (google this)
- it's something like, telling my router what to do with web traffic that's looking for my Pi? Not sure of the details.
buy a domain from cloudflare & give them the IP of my Raspberry Pi

Remaining Questions

How do outside packets know how to route themselves to my Raspberry Pi, i.e. how does the IP tell them that they need to get to my router and then to the Pi from there? How does that work? I know that the DNS service translates mywebsiteaddress.com into whatever my Pi's IP is, but what happens after that?

Tuesday (Raspberry Pi Setup Issues)

After the installation process had completed, the edges of the screen were being cut off and none of the settings in the preferences menu fixed it, I think the Pi saw the monitor as being bigger than it actually was??
I tried to poke around to see if there was a config file I had to modify to tell the Pi the actual resolution of the monitor, but when looking at the raspbian documentation it listed commands that didn't exist and couldn't be installed via apt (the package manager on raspbian)
I realized that the version of raspbian I was running was from 2022, and it probably needed to be updated to the latest version
I turned off the Pi, removed the microSD, put it into the provided "USB MicroSD Card Reader" and plugged that into my windows laptop
I installed all the recommended SD-card-formatting software but nothing was detecting the SD card, when I double-clicked the "E:" drive that had shown up it just gave me a message to "please insert a disc"
After looking around on the forums and getting frustrated, I ejected the reader and fiddled with the MicroSD card until I realized that I could push it in even further than I realized I could
Upon plugging it back in, everything successfully detected the card
I ran the "Raspberry Pi Imager" to install the latest version of the Raspbian OS on there
When I put the modified microSD card in the Pi and booted it up, it started the installation process over from the beginning but it detected the resolution flawlessly!!

Yippeeeee yayayayayayayayayay wahoo
I made a veryvery basic HTML file (using nano in the terminal) and was able to open it using Firefox, hell yeah!

Wednesday (Learning and Configuring Caddy)

I googled "caddy docs" and found https://caddyserver.com/docs/caddyfile
This seems to be what I need
Installed Konsole (a terminal emulator) on the pi because I like the look of it
Discovered that Konsole had weird formatting issues
Tried a different terminal emulator, Alacritty
Tried to edit the config file for alacritty but couldn't figure out how to make it work
Realized I was getting distracted
Opened w3schools to review how to write a basic CSS file
Created a CSS file
Found some nice colors from a palette a friend had recommended (catppuccino - Mocha) and colored the page using these colors in the CSS file
Moved the two files to /var/www/mywebsite/ as I was previously told to do
Next goal: configure and run caddy such that when I access https://localhost, I see my test website
I'm getting a warning when I run Caddy, there's some problem with how I'm formatting my Caddyfile but I'm not sure what
I'm having troubles and up until now I was just looking at the quickstart guide, maybe I should work through the longer Caddyfile tutorial
During the longer Caddyfile tutorial I was successfully having caddy respond with a string through http://localhost, but it only worked if I accessed it through the terminal command 'curl' - on Firefox it would automatically correct to https and then give me an error. After bothering that same earlier friend, they found a forum post that told me I had to tell Firefox to "forget" my previous visit to localhost during a previous step, and after that I was successfully able to see my test text in firefox
Ah. Nevermind. It's back to correcting http to https and the previous solution won't work. Shit.
For reference here's the previous page where we found the solution https://askubuntu.com/a/1127920
So much more troubleshooting occurred, god so much troubleshooting, I don't remember all the details anymore

Thursday

Finally just decided to run the commands from the caddy website to update caddy to its most recent version rather than trying to understand what they do in detail - the commands from https://caddyserver.com/docs/install#debian-ubuntu-raspbian
Connected to my router via 10.0.0.1 (I knew this address from when I had the friend over, she connected to the router via this address) and asked around to figure out what the username and password was so I could forward some freakin ports
When asking my friend about why it's 10.0.0.1, I ended up learning about how subnetting notation works, ie what "10.0.0.0/8" being reserved for " local communications within a private network" means (I had it backwards and thought the /8 meant the last 8 bits could be whatever you wanted, but its actually the other way around)
Googled "port forwarding" and ahhh now I get it, but hmm now I have the problem of needing to figure out what the LAN address of my Pi is
Figured out the LAN address via running ifconfig
Discovered that I could connect to https://localhost through chromium but not through Firefox, weird (I just had to click through a warning)
Set up a local site via my LAN IP!!! I can host a website on my Pi and view it on my phone or laptop!!!! (it gives me a warning that it's unsecure and it would only work while I'm connected to this wifi network but oh well)
Got told that this https-not-working-for-localhost problem shouldn't be an issue once I have an actual site, "https is for domains, not for IPs"
I bought a domain name through cloudflare!! aaaaaa, it's tikkyontheinternets.com
I tried installing the app the router requires but it turns out that I can't create an account without a special code rogers would have hypothetically sent me, so probably only the person who initially set this up can install the app and configure it, and the person that I suspect did that is currently in the UK. Hmm.
- (Also, even finding the right app was a pain - the router's configuration page told me to install the "Shaw Bluecurve Home" app but that doesn't exist, and looking through an installation pamphlet for the router it told me a DIFFERENT app name, the "Ignite HomeConnect" app which once again does not exist, and then I managed to find on the app store a DIFFERENT app "Rogers Xfinity" that apparently is the updated version of the Ignite app, and then I couldn't even access the app because I can't sign in. Fun.)
Next goal: I've learned that the "local" IP of my Pi is dynamic and can change, it's assigned via "DHCP", but I can give it a "static" IP by editing a config file or something. That is my next goal! ifconfig tells me that the subnet for my local IP is 10.0.0.0/24, so I'll set it to 10.0.69.69 or something, anything above 10.0.0.255 and below 10.255.255.255. just need to figure out how to do that.

Friday

I made progress on setting a static IP for the Pi: apparently the internet sources I was previously reading were talking about how to use dhcpcd to customize network stuff, but Raspbian uses nmcli. I've been reading the man pages for that, and it seems not too hard.
I made a big checklist for the process of making a nice looking website, if I get around to that.
Oh god the Shaw/Rogers account amalgamation is awful, it looks like we signed up for internet with Shaw and then they later got bought out by Rogers, I have access to the MyShaw app and an account with that but it's telling me that I need to also install the Rogers Xfinity app but I can't log into that app because no one has the credentials to it and I can't create an account with it because supposedly someone at some point is supposed to have received an email with an account ID but no one can find that

aaaaaaaaaaaa
I probably have to call Rogers customer support or something, this sucks

A week passes....

Got caught up doing some urgent life errands and an early Christmas party
Once I made the call on Thursday December 19th, I discovered that they can't let me do anything with the account unless the original creator is also there on the phone call with them, and he can't because he's in the UK until the 28th...

;-;
Ok, current goal is now to get all the non-technical things sorted before the 28th so it can be up and running asap

Friday December 20

This wasn't ENTIRELY necessary for the site to function, but today I decided to try getting static IP working
Took a while to figure out how nmcli works
Eventually discovered that nmtui exists, a GUI version of nmcli
It doesn't have all the same features though
What I had to do, which took me a while to figure out:
- clone the existing "connection" associated with my current WiFi network (a "connection" here is just an abstract list of settings for the computer to use)
- Edit the IPv4 settings to what I needed (method=manual, addresses=10.0.0.222/24, gateway=10.0.0.1)
Uhhh ok that's sorta working? Uncertain. (meta note, I don't remember what I was seeing at this time)
Friend tells me if it's letting me ping IP addresses but won't load webpages then I need to add in a list of DNS servers, she recommends 1.1.1.1 and 1.0.0.1
It all works!!

:D

Wednesday, January 8th 2025

The guy who initially set up the wifi is back!
I finally have access to the app for port forwarding, yayayayayayay (for feck's sake, it was a totally DIFFERENT but SIMILAR app and I totally could have logged in ages ago, [insert workplace-appropriate expletive])
For some reason the app kept saying that the Pi's Local IP was still 10.0.0.26 - when connected it was that, then if I disconnected the Pi from wifi it would go back to being 10.0.0.222 (the static IP I chose), then once it reconnected it would show 10.0.0.26. all this despite ifconfig showing 10.0.0.222 as my local IP.
Disconnected the Pi from the network and tried "forget device" in the app, once I reconnected it it finally showed the correct static IP I wanted :D
Port forwarding time
After some googling it looks like I need to specify port 80 in the port forward settings - my current understanding is that a packet will come to the router with a destination address of [IP of my Pi]:80, meaning something like "I'd like to connect to the HTTP service on the computer at this address", and then it'll send it to the Pi?
- (But - wait, does that mean that packets addressed to a different port won't be forwarded? Is that important? I do not know.)
Anyway: now, to set up the caddyfile correctly.
Ok I did a bunch of stuff with the help of my internet friend and now it's working!!!

:D
Had to tell cloudflare the IPv4 and IPv6 address of the pi (I didn't know it had an external v4 address, that's new! when I did 'curl icanhazip.com' I got an ipv6 address so I assumed that was the only external address for it
Had to set up port forwarding on BOTH port 80 and port 443 (for http and https respectively)
I switched to booting up caddy using systemd since that's what my friend is used to, should probably read the manpages on that

To summarize how web traffic gets to the Pi:

the cloudflare DNS server takes the request and translates "tikkyontheinternets.com" into the IP of my pi
the packet gets sent to that IP which gets it to my router (why isn't the IP enough to get it all the way there?? im confused again as to the mechanics of this, what port are other packets labelled with that get the router to send them straight to my computer when I request a web page??)
the router notes the port number and because it's the right one it forwards it onwards to the pi
the Caddy service running on the Pi sees the request and sends back the html/css file for the page

FINAL NOTES

things I learned

every device has an "external" IP and a "local" IP, one for use by computers on your local network (ie your wifi network) and one for use by computers outside that network.
to specify a range of IP addresses, you write 124.0.0.0/8 to mean all the addresses from 124.0.0.0 to 124.255.255.255, i.e. the /X means to keep the first X bits the same and to let the remaining bits be whatever. (unintuitively, this means that larger X's mean smaller ranges and vice versa)
DNS servers are "Domain Name Service" servers: computer sends human-readable address to server, server returns IP address. Cloudflare is doing that for my website, that's what I pay them to do (and somehow all the DNS servers around the world coordinate with each other)
DHCP exists: when you connect to the same network repeatedly, you may not always have the same address.
there are these mysterious things called "ports" that you address things to, and they may be the same or different from the linux "ports" like 'eth0' and 'wlan0' and 'lo' etc. Probably different. Not sure how they function
'nmcli' on linux uses abstract "connection" objects to store all the settings that you assign to the digital ports like 'eth0' or 'wlan' and they tell it how to connect to things. Internet stuff.
a "static site generator" is a program that turns some sort of markdown into static HTML/CSS. (I didn't use this but I considered it)
there's probably a bunch of other facts about the internets that I've just absorbed and forget that I now know, and the only way I'll later realize that a word or concept is meaningless to someone else is when I use it and they're confused and then I'll recall I learned it here

remaining questions

How do outside packets know how to route themselves to my Raspberry Pi, i.e. how does the IP tell them that they need to get to my router and then to the Pi from there? How does that work? I know that the DNS service translates mywebsiteaddress.com into whatever my Pi's IP is, but what happens after that?
Why isn't the IP of my Pi enough to get a packet all the way there?? im confused again as to the mechanics of this, what port are other packets labelled with that get the router to send them straight to my computer when I request a web page??

home