For this new year, I’d like to learn the skills necessary to self host. Specifically, I would like to eventually be able to self host Nextcloud, Jellyfin and possibly my email server too.

I’ve have a basic level understanding of Python and Kotlin. Now I’m in the process of learning Linux through a virtual machine because I know Linux is better suited for self hosting.

Should I stick with Python? Or is JavaScript (or maybe Ruby) better suited for that purpose? I’m more than happy to learn a new language, but I’m unsure on which is better suited.

And if you could start again in your self hosting journey, what would you do differently? :)

EDIT: I wasn’t expecting all these wonderful replies. You’re all very kind people to share so much with me :)

The consensus seems to be that hosting your own email server might be a lot, so I might leave that as future project. But for Nextcloud and Jellyfin I saw a lot of great tips! I forgot to mention that ideally I would like to have Nextcloud available for multiple users (ie. family memebers) so indeed learning some basic networking/firewalling seems the bare minimum.

I also promise that I will carefully read the manuals!

  • reinar@distress.digital
    link
    fedilink
    English
    arrow-up
    1
    ·
    11 hours ago

    Programming knowledge is largely irrelevant, as in to gain sensible benefits from it you have to be generalist software engineer with decade+ of experience of seeing it all. Then yeah, you can read any code, any stack traces and figure out the intent of developers of the system and what is undocumented/incorrectly documented.

    Focusing on one particular language is the right and wrong answer at the same time. Wrong in a sense that you’ll have to pick up other languages along your journey anyway and right because you need to achieve mastery in one of them to get to more advanced programming topics. Pick a language that you have fun using and don’t care about anything else.

    As for what to learn for self-hosting… Linux (pick a distro, let’s say ubuntu LTS w/o gui, ssh there and get comfortable with it. It includes installation, filesystems, RAID setups), networking, HTTP/S (that’s the main thing you’ll be interacting with as self-hoster and knowing various nuances of reverse proxying is a must), firewalling, basics of security and hardening, docker, monitoring, backups.

  • med@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    8
    ·
    4 days ago

    Lots of people have been talking about products and tools. It’s docker, tailscale, cloudflare proxmox etc. These are important, but will likely come and go on a long enough timescale.

    In terms of actual skills, there’s two that will dramatically decrease your headaches. Documention and backup planning. The problem with developing those skills is, to my knowledge, they’ve only ever been obtained through suffering. Trying to remember how to rebuild something when you built it 6 months ago is futile. Trying to recover borked data is brutal. There’s no fail-safe that you haven’t created, and there’s no history that you haven’t written. Fortunately, these are also the most transferable skills.

    My advice is, jump in. Don’t hesitate. The chops in docker/linux/networking will come with use and familiarity. If it looks cool, do it. Make mistakes. You will rapidly realise what the problems with your set up are. You will gain knowledge in leaps and bounds from breaking a thing vs learning by rote or lesson. Reframe the headaches as a feature, not a bug - they’re highlighting holes in your understanding. They signpost the way to being a better tech, and a more stable production environment.

    The greatest bit about self hosting for me is planning the next great leap forward, making it better, cleaner, more robust. Growing the confidence in your abilities to create a system you can trust. Honing your skills and toolset is the entirety of the excercise, so jump in, and don’t focus on any one thing to master or practice before hand!

    • Possibly linux@lemmy.zip
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      1
      ·
      4 days ago

      Networking is way more important than pretty much anything else. TCP/IP and http are going to stay for quite a while.

  • dutchkimble@lemy.lol
    link
    fedilink
    English
    arrow-up
    5
    ·
    4 days ago

    Dude it sounds you’re over skilled for the job. You just need to read some guides and you probably know already how networking works, very basic linux commands, linux folder structures, and then the concept of docker - primarily how it maps networking & folders from your “host machine” to the “docker container”, and how it loads services using a docker compose file. Especially for nextcloud, domain dns management and dynamic dns etc would be very helpful knowledge.

    Also, just a suggestion, chatgpt etc are super useful. You tell them what you want and it spits out custom instructions for your setup, and you’re able to counter question at any point. If it does make mistakes, which it will, it’s a learning opportunity for you to troubleshoot and figure out how everything works. All the best and if you have a question feel free to message me.

  • Human Crayon@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    7
    ·
    4 days ago

    Determination, patience, a willingness to learn anything you need to.

    If you have those, in time, you will be able to get your lab up and running. I started mine with a minimal knowledge of Linux (I could install it from a USB and poke around). Now it’s the center of my families digital life.

    You’ll get there in time.

  • rumba@lemmy.zip
    link
    fedilink
    English
    arrow-up
    6
    ·
    4 days ago

    Honestly, you just need to pick a video on follow along these days. There’s a load of YouTube videos out there that take you step by step.

    Lewis rosman recently put out a 14-hour mega video of doing everything, well he might have made some controversial choices, The outcome is quite comprehensive.

  • MXX53@programming.dev
    link
    fedilink
    English
    arrow-up
    3
    ·
    4 days ago

    The ability to read, and maybe watch a video. And then persistence for some of the trial and error you will run in to. All skills you need can be picked up with the above.

  • _cryptagion@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    1
    ·
    3 days ago

    Patience, most of all.

    Also, backups and notes. The solution you use to host might take care of the backups. For example, I use Unraid, so if any drive fails the system can simulate the data on that drive until I can get it shut down to replace it, and then recreate the data on the new drive.

    As for notes, those are important so that you can always know what you’ve done, and what you need to do. That way, if you ever have to do it again, say if you’re setting up another server or replacing one that failed, you know the steps you took to get it set up exactly how you like. It’s also handy because you’ll be doing things like assigning services to ports, and you’ll probably at some point want to know what services are on what ports without going through and checking each one. Things like that are handy things to stick in notes.

    Other than that, you don’t need a lot of skills to set something like a home server up. You just need to read the documentation for each service you’re planning to use, and get familiar with how it works.

    • TechieDamien@lemmy.ml
      link
      fedilink
      English
      arrow-up
      3
      ·
      2 days ago

      Unraid is not a backup. It is good, but if your data goes wrong for different reasons or you lose the entire device, you can’t restore it. Dedicated backups are a must for anything serious!

      • _cryptagion@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        1
        ·
        edit-2
        2 days ago

        Unraid absolutely is a backup. That’s the whole point of the OS. And furthermore , the backup can be backed up at any time and stored on another device, allowing you to restore the entire OS and its configuration. And by “lose the entire device”, I’m assuming you mean the OS is corrupted. At that point, you simply burn a new USB and reconnect the drives, or move them to any other system running Unraid.

  • Avid Amoeba@lemmy.ca
    link
    fedilink
    English
    arrow-up
    12
    ·
    edit-2
    5 days ago

    If you want to program something, the closest you’re gonna get to programming is Ansible and Bash scripts.

    You might want to get self hosting hardware like Synology or the like if you’re not ready to dig.

    Otherwise here’s some things you need to know:

    • Docker
      • Easy, consistent deployment of services in their own environments. Think a VM but with almost no overhead.
    • Docker Compose
      • Run docker containers with consistent configuration in files.
      • Connect various containers to each other on the same or different networks.
      • Get multiple containers to start together and talk to each other.
    • Systemd
      • Manage any service on Linux. If anything needs to start on boot, restart when crashed, start on timer, you want Systemd.
      • You can manage your docker compose containers lifecycle via Systemd.
    • NGINX/Apache/Caddy
      • A web server for reverse proxy. You’d probably need one at some point, especially if you want HTTPS. Your services get hidden behind it.
    • ZFS
      • Reliable redundant storage. You’ll need storage. Use ZFS with 2-disk redundancy.
      • Supports automatic snapshots for recovering from oopsies. E.g. deleted something or some software shat on your data.
      • Can use recertified disks from serverpartsdeals.
      • Can use USB disks or USB box with multiple disks. If you end up going the USB route, ask me for tested hardware.
    • Backup system
      • Something to do backup. There are many options.
    • Ansible
      • If you want to write code that describes your services and make them happen, you want Ansible. You write code (well YAML) and Ansible installs things, writes config files, sets up Systemd services, restarts things. It can be convenient especially if you have a lot of stuff and you want to be able to see all of your infrastructure in code in one place and be able to version it.
    • Prometheus
      • Monitoring your stuff. Is my backup service running? If not send me an email.

    Oh and use Debian or Ubuntu LTS.

    • Onomatopoeia@lemmy.cafe
      link
      fedilink
      English
      arrow-up
      3
      ·
      edit-2
      5 days ago

      Great summary!

      Why Debian or Ubuntu? (I have my own thoughts, but it would be useful to show even high-level reasons why they’re preferred).

      Re: Backup - Backblaze has a great writeup on backup approach today. I’m a fan of cloud being part of the mix (I use a combo of local replication and cloud, to mitigate different risks). Getting people to include backup from the start will help them long-term, so great you included it!

      • Avid Amoeba@lemmy.ca
        link
        fedilink
        English
        arrow-up
        4
        ·
        edit-2
        5 days ago

        Predictable cadence, stable operation, timely updates, huge community and therefore documentation. You can get up to 5 years from an LTS release of Debian or Ubuntu. With Ubuntu LTS and Ubuntu Pro (free) you could theoretically run a machine without upgrading for 10 years. If you run workloads in containers, it doesn’t matter how old the host OS is. As long as it’s security patches, you can keep on trucking.

    • LiveLM@lemmy.zip
      link
      fedilink
      English
      arrow-up
      3
      ·
      edit-2
      5 days ago

      If you end up going the USB route, ask me for tested hardware.

      Send these my way chief

      • Avid Amoeba@lemmy.ca
        link
        fedilink
        English
        arrow-up
        4
        ·
        edit-2
        4 days ago

        As briefly as possible:

        • Host side
          • If you use Intel, all is well.
          • If you use AMD…
            • Prior to AM5
              • Use an ASMedia PCIe USB card (StatTech, Sonnet)
              • X570 is especially bad, though I’ve had some success with B350, when using the chipset ports. The CPU ports are all bad. Small form factor PCs often only expose CPU USB ports. They work with single disk per port but if you peg a port with a multi-disk box, they crap out regularly.
            • Post AM5
              • Have only tested USB4 on X870 and it’s solid.
        • Client side
          • WD Elements / MyBook
            • If you get disconnects under load and you’re not on a shit AMD USB host, the USB-SATA controller is overheating. Open them and ahere a heatsink on it. Drill a hole in the case above it for better ventilation. Disconnections will stop. If you don’t want to deal with any of that buy the item below.
          • OWC Mercury Elite Pro Quad
            • Well built, solid controllers, no issues over a year of testing. I have 2, hosting an 8-disk RAIDz2 and 2 hosting a 5-disk RAIDz2.
          • Terramaster
            • A friend bought a 6-bay and tore it down for me. It has the same controllers as the OWC in a similar topology. If it’s cheaper it might be OK. I can vouch for the OWC though.
          • Cables
            • Get name brand cables, ideally higher spec than what you’d need! They aren’t important for a single USB disk but running a 4-disk box can max out the port bandwidth. If the cable can’t handle it… errors. Casually transmitting 10Gbps via easily detachable cables and ports isn’t trivial.
        • LiveLM@lemmy.zip
          link
          fedilink
          English
          arrow-up
          2
          ·
          edit-2
          5 days ago

          Much appreciated 🙏
          Gnarly stuff with the WD’s huh? Unfortunately I think that’s what I’ll end up having to put up with since I can’t really find the other options for a decent price around here.
          Funny enough I was half-considering just using a bunch of WD Elements. You think the MyBooks might fare any better?

          • Avid Amoeba@lemmy.ca
            link
            fedilink
            English
            arrow-up
            2
            ·
            edit-2
            5 days ago

            I used a mix of Elements and MyBook for years. Upon opening to heatsink, I didn’t see any significant differences between them. They use ASMedia or Jmicron, mostly ASMedia. The overheating issue depends on ambient temp and load. I’ve had one machine in a basement never experience them. Either way the solution is pretty straightforward and cheap. Once heatsinked, I haven’t had a problem.

            The cables they come with are good.

    • BrianTheeBiscuiteer@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      5 days ago

      Ansible is nice but I’ll repeat (as I said in another thread) it’s kind of advanced and gives a much better return on investment if you manage several hosts, plan to switch hosts regularly, or plan to do regular rebuilds of the environment.

  • yak@lmy.brx.io
    link
    fedilink
    English
    arrow-up
    5
    ·
    4 days ago

    Experimenting with VMs is the way forward.

    Basic networking knowledge is vital. And being able to configure your own firewall(s) safely is an important skill. Check out something like Foomuuri, or Firewald. Shorewall is brilliant for documentation and description of issues (with diagrams!) but it does not use the newer Linux kernel nftables and is no longer actively developed.

    Go for it with Nextcloud.

    I would also recommend at least having a shot at setting up an email server, although I would recommend pushing through to a fully working system. It is possible, and is very satisfying to have in place. The process of setting one up touches so many different parts of internet function and culture that it is worth it even if you don’t end up with a production system. The Workaround.org ISPMail stuff is a good starting point, and includes some helpful background information at every stage, enough so you can begin to understand what’s going on in the background and why certain choices are being made - even if you disagree with the decisions.

    Python is great for server admin, although most server config and startup shutdown snippets are written in BASH. You will no doubt have already begun picking that up as you interact with your VMs.

  • sugar_in_your_tea@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    12
    ·
    edit-2
    5 days ago

    Persistence and reading comprehension.

    There’s no need to learn Python or any programming language to self host stuff, you just need to be able to follow blog posts and run some Docker commands.

    I’m a software dev and haven’t touched a single line of code on my NAS. Everything is docker compose and other config files.

  • sunstoned@lemmus.org
    link
    fedilink
    English
    arrow-up
    8
    ·
    edit-2
    5 days ago

    if you could start again in your self hosting journey, what would you do differently? :)

    That’s an excellent question.

    If I were to start over, the first thing that I would do is start by learning the basics of networking and set up a VPN! IMO exposing services to the public internet should be considered more of an advanced level task. When you don’t know what you don’t know, it’s risky and frankly unnecessary.

    The lowest barrier to entry for a personal VPN, by far, is Tailscale. Automatic internal DNS and clients for nearly any device makes finding services on a dedicated machine really, really, easy. Look into putting a tailscale client right into the compose file so you automatically get an internal DNS records for a service rather than a whole machine.

    From there, play around with more ownership (work) with regard to what can touch your network. Switch from Tailscale’s “trusted” login to hosting your own Headscale instance. Add a PiHole or AdGuard exit node and set up your own internal DNS records.

    Maybe even scrap the magic (someone else’s logic that may or may not be doing things you need) and go for a plain-Jane Wireguard setup.

    • jg1i@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      5 days ago

      For sure use Tailscale for VPN. They have apps for iPhone, Android, macOS, and Linux, so setting up your own personal network will be easy. Hosting on the real internet is definitely advanced and not always necessary.

  • JackAttack@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    5
    ·
    edit-2
    5 days ago

    I think if you have a mini PC or something you don’t mins installing server specific OS on, proxmox VE is pretty great. It can be a bit of a headache depending on what you want to do but it also makes expanding your self hosting super easy/fun.

    Proxmox let’s you created virtual machines or containers and easily manage them.

    Currently I run a mini PC.(Beelinks are great for this and pretty cheap with more power than a raspberry pi)

    1. Proxmox VE as the operating system
    2. Jellyfin as my media server
    • looking to install home assistant and some others as well.
    1. Tail scale to use as a VPN into my network privately from anywhere.

    Simplest set up? No. But I also just started self hosting. You’re gonna hit some.headaches regardless but it’s also extremely rewarding and I’ve learned alot.

    Edit: if you’re only wanting to run one thing (say jellyfin) . A raspberry pi with tail scale and jellyfin is how I started. Works just fine.

  • iAmTheTot@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    6
    ·
    edit-2
    5 days ago

    I self host many services without any coding languages under my belt. I use Unraid because I found it user friendly for a newb like me. The most important skills I needed were good data backup habits just in case I messed something up, a willingness to read and learn, and the persistence to try more than once.

  • dontblink@feddit.it
    link
    fedilink
    English
    arrow-up
    5
    ·
    edit-2
    5 days ago

    Enough focus to read documentation.

    That’s really it. If your purpose is just self hosting learning bash could also be helpful. And yeah Linux would be a great choice.

    But mostly, if you want to self host an instance of Nextcloud correctly and without having to deal with too many unexpected things, you have to read the documentation and do not rush. Most self hosted stuff isn’t “install and use”, because you’ll be your own server manager, and everything requires attention to be managed.

    Docker or not docker you will have to deal with configuration, settings, requirements and updates.

    So understanding how to read the docs/search and open github issues and taking time to read everything would be the most important skill for me.

    Also writing down what you are doing would indeed be helpful too, in order not to lose track of what you’re doing on your server. (Check out Ansible).

    Most apps out there simply need you to know about permissions, systemctl services and package managers.

    Try to always find a specific package for your distro for everything you install (eg. .deb for Debian), and have strategies when this is not possible (aka using a Python venv when installing python programs).

  • tehnomad@lemm.ee
    link
    fedilink
    English
    arrow-up
    3
    ·
    5 days ago

    One thing I would do differently is setup LDAP and OIDC so you can use the same authentication credentials for different apps (at least the ones that support them). I use LLDAP and Authelia for this purpose.