Someone on Lemmy posted a phrase recently: “If you’re not prepared to manage backups then you’re not prepared to self host.”
This seems like not only sound advice but a crucial attitude. My backup plans have been fairly sporadic as I’ve been entering into the world of self hosting. I’m now at a point where I have enough useful software and content that losing my hard drive would be a serious bummer. All of my most valuable content is backed up in one way or another, but it’s time for me to get serious.
I’m currently running an Ubuntu Server with a number of Docker containers, and lots of audio, video, and documents. I’d like to be able to back up everything to a reliable cloud service. I currently have a subscription to proton drive, which is a nice padding to have, but which I knew from the start would not be really adequate. Especially since there is no native Linux proton drive capability.
I’ve read good things about iDrive, S3, and Backblaze. Which one do you use? Would you recommend it? What makes your short list? what is the best value?
You must log in or register to comment.
Restic (or rustic) and Hetzner storage box 🤩
A server in a friend/family member’s home. All of the cloud based backups I’ve encountered seem either unaffordable or have annoying limitations.
This is the way
This. Install a NAS in a friend’s house, give them 10% of the capacity as a thank you.
I have one in theirs, they have one in mine.
100% this. OP, whatever solution you come up with, strongly consider disentangling your backup ‘storage’ from the platform or software, so you’re not ‘locked in’.
IMO, you want to have something universal, that works with both local and ‘cloud’ (ideally off-site on a own/family/friend’s NAS; far less expensive in the long run). Trust me, as someone who came from CrashPlan and moved to Duplicacy 8 years ago, I no longer worry about how robust my backups are, as I can practice 3-2-1 on my own terms.
Or just simply just sneakernet drives to a friend’s home. Good excuse to visit a friend more often.
Yup I’ve got a box in my mum’s house that all my off site backups go to and it’s a damn site cheaper just to give her some money for the electricity cost of it each month than pay for any cloud service.
3,2,1.
My nas is a Synology with raid.
- Backup with versions to a single large HD via USB. This ransomware protection or accidental deletion. (Rsync)
- Offsite copy to backblaze b2.One version. (Rsync) (~$6/month) This would be natual disaster protection. flood, fire.
- Second not raided cheaper Synology at a friends on the other coast. This has ~3 versions. Sorta the backup to the first two.
3, 2, 1. ❤
Without implementing this, it’s a delusion that some company, regardless of the size and reputation, can be trusted to keep our data safe.
Also don’t forget to restore test, otherwise you may as well not do backups. I have a reminder for once a year to test them, not just if it works but also what the performance is just in case.
This is the part that gets me. I don’t know how to automate this. I periodically retrieve something from the backups, which, so far, has worked. That’s not really good insurance, though. Any suggests or resources, ideally for borg and/restic?
You can get append only backups on backblaze with their lifecycle rules. So that can have ransomware protection too
“Append only backup” what’s that?
Its a system where you can only apppend, not delete.
https://en.wikipedia.org/wiki/Append-only
Its what’s required for ransomware safe backup system, since the attacker can’t delete your backups because they can only append
Oh, I see, I didn’t know that “nomenclature”. Thanks! Good for some thing, dangerous for other because the stored data keeps growing.
With backblaze you can set lifcycle rules. For example, any file with the regex “daily” in it automatic gets deleted after 30 days. And any file with “yearly” in it gets deleted after 5 years
How do you do versioning with rsync? I use rdiff.
I use the unlimited consumer backblaze with private key on a windows VM. I provision a 40tb iscsi connection to the VM from a NAS and all kinds of various homelab systems and devices store thier backups there. Works great and is the cheapest possible option at $9 a month.
Is that not against their TOS? Could make the service more expensive for the rest of us
I’m not sure about the iscsi protocol. They allow VMs, including harddrives via USB, so the point of doing this making it more expensive does not apply considering someone could just hook up 100tb+ of USB drives and still be clear under the TOS.
If they did have a problem with this I would just do that instead.
Bsckblaze doesn’t care, they know they’ll get their money when @Unforeseen@sh.itjust.works tries to get data back from backup.
Restoring data is free from backblaze.
Well yes and no. The rate at which you get your data back is where the gotcha lies anything up to 8TB is free if you send them $280 and they’ll refund the money once they get the drive back. Anything over 8TB is where it gets pricey.
And do that multiple times?
There aren’t any “gotchas” they absolutely lose money us who store more than a few TB but its worth it considering that we are in the minority.
Someone from BB posted a graph showing the distribution of data usage over all users and the VAST majority are under 1-2 TB
And the situation where I need to restore more then 8tb would be when I lost all my original data, and the backup NAS itself.
If that happens I’m not worrying about spending $280.
I use restic to backblaze b2.
Yep, Duplicacy to Backblaze B2 for me
Same
I’ve been using Restic to Backblaze B2.
I don’t really trust B2 that much (I think it is mostly a single-DC kind of storage) but it is reasonably priced and easy to use. Plus as long as their failures aren’t correlated with mine it should be fine.
I quadrupal vote for this combination.
You could trust B2 more; maybe dig into their structure. They’re solid, and not only that they provide an awesome service with their yearly HD failure rate evaluations, in which they describe the structure of their data centers.
I’m terms of NPS, I’m on their side. Unless something comes out and shady business practices, I’m brand loyal to B2. Been with then for years, and love the service, pricing, and company.
I think it depends on your needs. IIUC their storage is “single location”. Like a very significant natural disaster could take it offline or maybe even lose it. Something like S3 or Google Cloud Storage (depending on which durability you select) is multi-location (as in significantly distinct geographical regions). So still very likely that you will never lose any data, but in the extreme cases potentially you could.
If I was storing my only copy of something it would matter a lot more (although even then you are best to store with multiple providers for social reasons, not just technical) but for a backup it is fine.
I think I see what you’re saying.
B2 has multiple data centers around the world - at least 3 in the US and 1 in EU, that I know of. If you want your data replicated, you have to create buckets in multiple locations and connect them for replication, which they’ll do for you (the replication).
If you’re saying that they don’t automatically store multiple copies of your data in multiple locations for you, for free, you’re right. But they do have multiple data centers located around the world, and you can create multiple buckets and configure them for automatic replication so you have redundancy. You have to pay for the storage at each replicated location, though. If you want a bucket in Sacramento, it’ll cost you those pennies. If you want it replicated to Rest on, you’ll pay double the pennies. If you want it also replicated to Amsterdam, triple the pennies.
I don’t think it’s fair to say that they’re single location that could have a natural disaster and you therefore lose your storage. It’s only like that if you set it up that way, and it’s pretty trivial to set up global replication - it just costs more.
That’s true. And I’m not saying B2 is bad, it is just something that you should be aware of.
Their automatic replication isn’t quite as seamless as GCS or S3 though. For example deletes aren’t replicated so you will need a cleanup strategy. Plus once you 2x or 3x the price B2 isn’t as competitive on price. My point is that it is very easy to compare apples to oranges looking at cloud storage providers and it is important to be aware.
For me B2 is a great fit and I am happy with it, but I don’t wan to mislead peope.
Great clarifications!
Enabling multi DC redundancy is really easy though. The other providers you mentioned may have it by default, but they’re also a lot more expensive.
I love that they let me pick my own redundancy strategy, without forcing me to pay for theirs
My current strategy might be a bit over the top,but it works.
I have two main entities that contain data worth backing up - the NAS and to a much smaller extend my Proxmox cluster (which is partly within my house,partly at Hetzner).
User PCs do not have any User data saved, they all work with network drives mapped to the NAS, only irrelevant amounts of data are stored on them that gets backed up via Free File Sync. For the Notebooks I use the same concept as we are using a WG VPN 99,9% of the time anyway,but some important folders get also synched via Free File Sync for offline use if no mobile connection is available.
For proper backups I have basically three classes of data that I maintain: Prio 1: The real real important stuff. Photos of once in a lifetime events, important documents, etc. Prio 2: The stuff you still don’t want to loose. All other photos, the scanned documents, home folders, VMs/LXX backups, configurations, etc. Prio3: Everything else,mostly data that could be downloaded again. Easily. Movies, etc.
Prio3 data is currently only living in the NAS and does get backed up once in a while on a external hard drive. It’s mainly backed up as I am lazy and in case the NAS craps out I don’t want to reload all the stuff…that would take months.
Prio2 data gets backed up fully: For the NAS data: It gets backed up to B2 with versioning according to my needs (usually 3d,2w,3m,1y,but that depends highly on the source). Additionally full external hard drive backups every few weeks. (I would kill to get my hands on a proper tape drive again,I had one back in the day,but it was used and old and died) Some data is also stored on Synology C2 atm,but I will replace that soon with another cloud provider, likely Hetzner.
For Proxmox: Basically the same, but I use TUXIS instead of B2 and Hetzner instead of Synology C2. Additionally I have a old PC with Proxmox backup server which turns on once a week and safes the whole cluster before turning off again. In the future this PC is planned to replace the External hard disk’s,but currently hard drive prices are insane.
For the P1 data: Same as above,but it’s definitely staying on a second cloud provider. Additionally I also create archive blue ray disk’s every few month. (Usually every 4). These go into the safe deposit box at my bank and additionally to a second storage location.
And of course I have detailed instructions about this in my will so even if both my wife and I die my kid can figure it out.
Backblaze 200% of the time.
The only thing that sucks about backblaze is that they’re not designed for enterprise. No account balances. No multi users.
It have a free service?
Well, no, but I use it for free.
Because backblaze doesn’t let you maintain an account balance, I almost had all my data get deleted one time my credit card false-positive blocked the payment (for “my protection”).
I ended up getting a credit card specifically for B2. I use it for nothing else.
Turns out some credit card companies dont charge you anything if your bill for the month is <$1. So, yeah, I accidentally get backblaze for free.
For devices like laptops and PCs. I use Urbackup to make backups.
For all the apps I host on Kubernetes I setup S3 backups to self hosted Minio.
I like S3 because I only pay for what I use and it has auto storage tiering.
Timely post.
I was about to make one because iDrive has decided to double their prices, probably because they could.
$30/tb/year to $50/tb/year is a pretty big jump, but they were also way under the market price so capitalism gonna capital and they’re “optimizing” or someshit.
I’ve love to be able to push my stuff to some other provider for closer to that $30, but uh, yeah, no freaking clue who since $60/tb/year seems to be the more average price.
Alternately, a storage option that’s not S3-based would also probably be acceptable. Backups are ~300gb, give or take, and the stuff that does need S3-style storage I can stuff in Cloudflare’s free tier.
Yeah, it was $2.5/tb/month, now it’s $4.1/tb/month.
Still cheaper than backblaze’s $6 which seems the only other option everyone suggests, so it’ll have to do for the moment.My idrive plan went from just over $100 to $250.
I created another account, paid for another year at a promotional price, and then deleted my old account.
I will eventually have to come up with a more sustainable cloud/off site backup now that i need more than just a few TB.
Since this is really my “last resort” backup, I’m not too concerned, as anything that would require me to actually restore from this backup set would likely be catastrophic in a life-ending way.
I use Backblaze B2 through my Synology NAS to offsite my important data. Most things though I just backup locally and accept the risk of needing to rebuild certain things (like most of my movie/TV media files since I can just re-rip my physical media, and the storage costs are not worth the couple of days of time in that unlikely case).
I really think this is key when thinking about your backup strategy that is specific to self hosting compared to enterprise operations. The costs come out of our pockets with no revenue to back it up. Managing backups for self hosting IMO is just as much about understanding your risk appetite and then choosing a strategy to match that. For example I keep just single copy in B2, since the failure mode I’m looking to protect against is catastrophic failure of my NAS which holds my main backups and media. I then use Proton Drive and OneDrive to backup secrets for my 2FA setups and encryption for my B2 bucket. This isn’t how I would do it at work (we have a fair more robust, but much more expensive setup). But my costs for B2 are around $15/mo which I am fine with. When I tried keeping multiple copies it had grown to over $50/mo before I cared enough to really rethink things (the cost of the hobby I told myself).
I use restic with a wrapper script to automate it on all of my machines. The backend storage can be anything that speaks S3, so B2, or iDrive would both work. I currently use Storj for my backend. It’s globally distributed storage, so no single point of failure geographically and it’s cheap. Backblaze is also a great company, but I’ve grown a little skeptical since they went public.
I’m on Pcloud, server with rsync+rclone to move files from file system to cloud and use it as a unified file system.
The lifetime storage offer from pcloud has been worth it for me and I even upgraded it from 2 to 12 TB
I use borgbackup, with daily backup to borgbase.
At some point I want to set up a distributed file system between multiple locations as both a backup target and also a network share with automatic snapshots or some other undelete mechanism, but I still need to get the hardware for that and the current setup works well
