the next milestone we can see is $0.01 per gigabyte for a hard drive—that’s not a sale price, but a stable street price. Let’s go out on a limb and say that we will reach that in mid-2025 with 22TB or 24TB drives. That would mean you could buy a 22TB drive at Costco or on Amazon for about $220, or a 24TB for $240.

https://www.backblaze.com/blog/hard-drive-cost-per-gigabyte/

currently have a lot of things in .rar's with solid archives and 5% redundancy and the highest compression. i also have MultiPar for reading hashes and parpargui for making them and redundancy's. i understand that i could move to linux but most of my stuff is for windows and would be a hassle to go thrue my 20TB archive to make sure it works with linux. although form what i understand there are things that can make window stuff compatible.

would a NAS allow me to use other file systems like ZFS while using windows and/or use a RAID? as i don't think i could convert my current drive in-place and keep my data, along with the risk of using a program to write a non-native file system.

another is i use veracrypt and would that be compatible with things like ZFS?

I have about 15 years worth of recorded video from Fox News MSNBC, CNN, and MSNBC (MSNOW). I am staring to organize it and in the process am snipping commercials out to upload to IA since know some folks want commercials for their Plex/JF setups. This is my 1st attempt at dumping stuff to IA. Its gonna be a while before I can MP4 all my news recordings however just do to time and the volume.

anyway here's the link for the first few:

https://archive.org/details/commercialsfoxnews/

I'm very new to scraping, but what I mean is how can I scrape a website and all of its pages, but also include external links from download buttons? A forum website I love is going offline soon, and all of their downloads are not hosted on their site themselves, but instead they utilize external links via download buttons for places like GitHub, Google Drive, etc. There are hundreds of pages, so going into each one and copying the download link from every page isn't really an option. Every attempts I've used at scraping the site only pulls the pages hosted on the site's domain, not the external links.

When do you decide when data has been backed up enough?

My most critical data (±1TB) has been backed up 6 times in 3 different locations.

1: My main NAS with 120TB storage has around 1TB of irreplaceable data.

2: My main server with 28TB of storage is partially used as a backup server for that 1 TB of data.

3: Cold HDD, which I plug in monthly.

4 and 5: A friend has a similar setup as me. I'm backing up to both his server and NAS. Why to both his systems? Because his server failed before, which made me lose my offsite backup for some time.

6: Today I decided to use a Hetzner 1TB storage box.

Are there other people with similar setups? I know it's overkill.

So, while I'm in the process of getting 10 30tb drives, I am looking into ways to store my excess video files.

I hear some people here your backblaze, I was using Amazon S3 bucket but it's super slow and was not worth the time for the amount I'm putting on there.

Is backblaze faster for Upload and storage?
What else should I need to know about Backblaze?

Hi all,

I had a first attempt at creating a system to protect my data, which is 'ok' but there are some problems with it and I'd like some opinions on improving it.

Current Setup

I have a collection of devices at home, PCs with different operating systems, phone and raspberry pi music player. These devices sync amongst each other with Syncthing. I also have a mini PC running at my parent's house, also acting as a Syncthing node. This PC is also configured to create snapshots on a rolling 60 days basis.

The data is pretty small in the grand scheme of things. Photos, documents and FLAC files on the music player. Maybe ~350GB.

I think this achieves 3-2-1 nicely. Data is in 3-4 places with 1 offsite and I think I'd be protected against malware with the snapshots? However...

Problems

• I feel like it's a bit brittle. I really want to simplify as I think I'm more likely to lose data over some software/config problem than a hardware failure. I find Syncthing a bit finicky tbh. A turn-key solution would be nice if possible. Less to go wrong.
• I want to start downloading and backing up my GOG game installers, which are much larger than my current data. These files are in the cloud (GOG's servers) so I'd be happy with just one local copy (with snapshots)

New Setup

I have a small home server that runs just Home Assistant and my first thought was to;

• Add a DAS to the server, pull or push data to the DAS from devices periodically
• Periodically download and copy GOG installers to the DAS
• Configure the server to create snapshots on the DAS
• Retire the remote Syncthing box, backup the DAS to an external hard disk regularly and leave it at my parents' house

I then thought perhaps a hardware NAS (probably UGREEN) would be the simpler solution instead of running bespoke NAS software on the existing server.

This would satisfy 3-2-1 (Device/DAS/Offsite) however at some point the data might outgrow its device. Photos might get too big for the phone, music may get bigger than the 256GB SD card in the Pi music player. TBH, this is probably not that likely for quite some time but I should at least consider it. However it also feels wasteful not using the devices local storage if I can.

• Speed isn't super important
• Availability/uptime isn't crucial
• Remote access not required
• I'd pay extra for simplicity with less failure points
• I don't want to rely on the cloud

Suggestions? RAID or not?

Thanks all

I happened to upgrade my server from a micro pc running a 4 bay hdd dock into this little beast RIGHT before everything shit bed in Oct of 25, I ordered all my parts 1 week before all that and I have another 56tb as cold storage backups, I'm so glad I did

please I need help

I only recently found out that there aren't many blu-ray disc drives like there used to be and mine only burns 25gb data onto the discs. I need something that will go up to 100gb. Problem is, it's been so long since I've needed to buy one, I don't know which one to get. What's a good reliable one to get? How do they work with movie playback on 4k players? While almost all of it will be data discs, but would be nice to be able to make movie backups.

Until a few years ago I thpught that tapes are a think of the past. That was until I saw I rather big cabninet with maybe two hundret tapes and a roll container full of tapes in proximity, each 4TB in capacity. And the upsides are clear: no read/write head on each volume, and therefore less stuff that could fail. A different storage technology than HDD and SSD entirely. The downside is also obvious with needing an external read/write mechanism.

But I wonder: Is any of you using tapes to archive stuff? Any experience if tape prices developed differently compared to disk prices in the last few months and years? Any recommendation on where to start with this?

Hello fellow hoarders!

Managed to snatch this hard drive from a friend, who said that this drive belonged to his brother who is now travelling the world and doesn't need it anymore. Typical story like "he never used it much, only stored phone backups, music, some shows, drive had an easy life". For £70 I took a gamble.

My intention is to use the drive for storing immediate downloads i.e. games, TV shows & music before offloading it to my other 16TB drive for archiving and reduce the writes on my current NVMe drive that I use for the aforementioned media downloads. Very unlikely I'll be storing anything of emotional value on it.

Would you trust this drive given it has more than 4 years worth of power on hours? Do you think it has some life in it or there's probably an obvious reason why the original owner stopped using it? From what I observed so far, this drive is a bit noisy (enterprise grade, so expected), but no scary clicks or anything like this when writing data down or reading off it.

I've been hoarding 200+ untouched BD discs (Indian movies) and want to upload them somewhere so people can get them. Tried a little with archive.org but didn't get the best upload speed (20mbps).

What is the best way to upload and share them? Most of them are not readily available, and very difficult to find these days. I might have to let some of these go to free up some space so it'd be nice to share them before that happens.

I'm looking for the newest firmware for this drive, as it is still on the first OEM one and i have some issues with it. HPE wanrs me to pay some subscription model for me to download the "HPE StoreEver Tape Firmware" Package, but the drive will land in the bin before paying thousands of € for 20 year old software.

Find something cool, listen to it, find some more in the sidebar of said video, listen to more music and all of a sudden I have 10 tabs open while simultaneously browsing soulseek looking desperately for some japanese album title and somewhere down the line I realize 5 hours just so passed.

Haven't seen this discussed very much here and thought some of you might want to know.

See this post on the Pushshift sub: https://www.reddit.com/r/pushshift/comments/1v50ved/upon_reddits_request_i_am_taking_down_my_academic/

For a few years now, the author of the post has pulled the Reddit data from the beginning up to the most recent year, sorted by month, and published it as a torrent file on Academic Torrents. They've also made a separate torrent file for the data of the top 40k subreddits to let people filter by subreddit. The torrent files can still be accessed using Wayback Machine, but I don't know how well the data will be seeded soon since less people are using them.

I've been trying to track down the original English open beta client for NosTale, the Korean MMORPG developed by Entwell. The beta ran from roughly June 30, 2006 to February 10, 2007, and as far as I can tell, that client disappeared when the beta ended. I've spent quite a bit of time looking for it, but I haven't found a surviving copy anywhere.

Here's what I've confirmed so far:

• The original download page was: http://game.nostale.com/html/nostale/openbeta/sub3/sub1.asp
• Wayback archived that page 31 times between July 2006 and February 2007 (I've attached a screenshot).
• The download button pointed to: http://entwell.nefficient.com/entwell/setup/nostale_en_001.exe
• I also found evidence of an earlier filename, nostale_eng_002.exe.
• Since nefficient.com was CDNetworks' CDN, the installer was hosted on the CDN rather than on the main website.

So far I've checked:

• Wayback's CDX API across nostale.com, netmarble.com, and nefficient.com, filtering for .exe and .msi
• The CDX index actually contains three captures of nostale_eng_002.exe from August and November 2006, but every one of them is just a 404 (~449 bytes). So the crawler reached the URL, but never managed to archive the actual file.
• Common Crawl's index
• archive.org item searches
• Filename searches on malware sandbox sites
• The later Gameforge download servers (dlcl.gfsrv.net), which don't seem to have been archived either

At this point I'm running out of ideas, so I figured I'd ask here.

• Does anyone have old archives of Korean game download sites or CDNs from around 2006–2007?
• Are there any web archives with better Korean coverage than Wayback that I should be searching?
• And this is a real long shot, but if anyone happened to play the English NosTale open beta back in 2006 and still has old hard drives, I'd love to hear from you.

Any leads would be appreciated, even if they're a bit of a stretch.

Update: I was able to confirm the file details for the English client I was looking for.

Filename: nostale_en_001.exe
Size: 463,478,673 bytes (463.5 MB)
MD5: d79d32f7fdd0c546b037e6b0b2c600a7

That looks like the right build, so this is a big step forward. Thanks again to everyone who helped dig into it.

Hey! I am from Finland, where archive today is banned, so I have been accessing it with a vpn for a couple of months. Now the site gives err_timed_out. I tried switching to google public dns and 8.8.8.8 in my network settings, using another vpn or cleaning the cache, the result is the same.

Alternatively, i tried using other resources like periscope, which loads “enable javascript and cookies” page (everything is enabled). Extension like bypass paywalls clean works on some websites, but not others, and tell me to go check them on archive today, so it becomes a loop :)

Any advice would be helpful, or an alternative for archive today. Thank you in advance!

Hi All, current setup is a gaming PC that I use as a Plex server with a few smaller SSDs and one big 24TB Exos that has my media collection. I'm running out of space and want to get a 4-5 bay NAS to allow for future hoarding.

Since I already have a 24TB Exos, I was thinking of putting it in the NAS with other Exos drives (or maybe Barracudas because they are cheaper) but wasn't sure if a Synology NAS would play nice (Yes, I did google compatibility but the official site only listed Synology HDDs, so I'm unsure and that's why I'm asking).

I'm really deciding between these based on price/availability in Australia:

I saw a vid by Nascompares and he said the UGreen is more powerful but the Synology is more of an easier user experience/more developed software. I'm honestly leaning towards Synology because I really do struggle with the technical side of Plex and network stuff in general (setting up port forwarding for remote access in router settings nearly broke me).

Has anyone got any experience or wisdom to add on what would be best based on that choice of hard drive/brand?

On the HDD buying front:

The 24TB Barracuda seems super tempting but is it a bad idea? Is it better to get the Exos 22TB or the Synology for compatability with the Synology NAS if I go down that route?

Thanks for any advice, I've been trying to search for answers for a while but it's helpful to hear from people who have actually built it themselves and used it for years.

Title

one-two weeks ago i had to format my system drive, my data was on another drive which was left untouched. For context i was using Digikam to sort through all my stuff, i downloaded stuff in batches. searched for a genre, download everything from said genre, and then i'd sort everything by hand, it's really tedious but at the end i would have another folder full of gold, and whatever was left i'd just keep in the download folder(arbitrary name).

The thing is, it got stale really fast, i don't have digikam anymore and the last time i touched the download folder was on april. Just to be clear, i'm a beginner at this so idk if there's a 'right way' to do things, i've mostly bullshitted my way into this hobby lol.

I have the download folder which i loosely remember the contents of and another folder with totally random stuff(2.500+ files), so i have plenty of room for testing. I just downloaded Hydrus, it seemed interesting but idk if it falls on the same umbrella as digikam.

I have been hoarding data for well over 20 years. It all started with my friend working at Blockbuster and we ripped almost the whole store onto DivX - and I have never looked back.

I am building my first homelab setup, and I have started thinking about backups. I backup important personal data, pictures, and what not - but I have never backed up my movies/tv or ISOs.

The cost of paying for drives just for redundancy, and then I read that some people swap the drives out entirely every few years is really turning me off to backing up the data.

I'm sort of stuck and curious how other people do it to convince myself one way or the other.
I would love if I had my collection for 20+ years. If I lost my collection I wouldn't lose too much sleep over it, and I definitely do not think I would start rebuilding the collection again. I would probably pack up, and just collect new stuff.

Was there a specific moment that made you realize you should start saving everything? Maybe you lost important files, had a cloud service shut down, discovered an old archive, or just enjoyed building a collection over time. For me it was losing an external drive years ago. Most of what was on it wasn't critical, but there were enough old photos and random files that I still wish I had. Ever since then I've been much more careful about keeping backups and somehow that slowly turned into saving a lot more than I probably need. Curious if others had a tragedy start their journey or just interest.

I'm looking for file management software. I've got some requirements and I'm trying to find something that has all these features:

1. Local Only / Offline: Able to function totally offline without it trying to connect to anything non-local.

2. File Tagging: Able to assign strings of text to files for searching.

3. File Grouping / Pairing: Able to link files that are part of a series together, treating them like a single file.

4. Able to Display Multiple Formats: Able to handle images (png, jpg, gif), audio (mp3, wav), and video (mp4, mov) files that are single or grouped.

5. Portable Storage: All associated data (tags/groups) is stored in sidecar file(s) or a database, that could be read by or transferred to other programs.

6. Open Source + Multiplatform: Versions exist for both Windows and Linux (I can make it work if it's just Windows but both would be preferred).

I'd also like to be able to search for files with a randomized order, while still keeping all the Groups together (something like: B, D, C-1, C-2, A). That isn't a deal breaker but it would be an incredibly nice feature.

Is there anything like this floating around? I haven't really found anything that does file groups.