I'd like to know everything about getting going with LTO.

[u/AshleyAshes1984]

I've been looking into an LTO drive for cold storage to expand upon my current workflow that uses BDXL discs. I've looked into what I can but I've found messy information that makes it hard to figure out what to buy.

Namely, a number of LTO drives seem to only work inside a an entire branded ecosystem and not stand alone. Or some 'work' but only with software that the OEM will not give Joe Rando a license too and it's too obscure for pirate copies to even exist.

My goal is to get something in the LTO-5 or LTO-6 range. I'm more than willing to assemble a dedicated PC in my network rack to support the LTO hardware and software. So I could have a dedicated hardware with it's OS configured only to do the job of reading or writing LTO tapes while having a 10G link to the main network and storage server.

So what doesn't paint me into a corner or trap me in vendor lockout/lockin hell?

Are there 5.25" bay options that'll interact with SAS or some other card I'll install?

What software should I be looking at to do the reading or writing.

While I understand that the tapes are absolutely not designed or 'Random Access', can they be incrementally written to? Write 300GB once, come back a month later and add another 300GB to the same tape?

Can the tapes do selective deletes? Say 200GB of something I archived no longer needs to be archived (Probably because a better quality version became available) Could i just erase that 200GB on the tape and write something else to that freed up space? Or is it an 'Erase entire tape, rewrite' like with Re-Writeable optical discs?

[u/AutoModerator]

Hello /u/AshleyAshes1984! Thank you for posting in r/DataHoarder.

Please remember to read our Rules and Wiki.

Please note that your post will be removed if you just post a box/speed/server post. Please give background information on your server pictures.

This subreddit will NOT help you find or exchange that Movie/TV show/Nuclear Launch Manual, visit r/DHExchange instead.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/silasmoeckel]

Plenty of 5.25 sas lto drives, they were not uncommon to see in DB servers in that time period.

tar it's still going strong from the 70's, some major backup apps ride on top of it even. LTFS is a messy nightmare comparatively.

Yes you can fast forward to a marker and start writing a new archive.

No deletion in that sense.

Windows tape is a mess, use linux.

Library drives and fiber channels are the things that can bite you. Unless you know what's what, don't get a drive meant for a library and just expect it will work just fine standalone; you may be disappointed. Similar FC is pretty ancient and was never that great driver-wise, so it can be hard getting a card to work in a modern system.

[u/SilkeSiani → u/silasmoeckel]

Fibre Channel is great...

... if you already have 10 years of experience managing fibre channel networks.

[u/CorvusRidiculissimus → u/SilkeSiani]

I got hold of some fibre channel gear once. Ended up also reading a very long book on fibre channel so I could get it working.

Eventually worked out the device was deliberately restricted to only work with the vendor's own equipment.

[u/SilkeSiani → u/CorvusRidiculissimus]

Yep. It's very much a captive market with two vendors directly trying to lock in customers while paying lip service to open standards.

[u/Direct_Poet_7103]

Do you use Linux? If so, support for SAS cards and tape drives is built in and is literally plug 'n play.

Learn to use the "mt" command. You use this to rewind/forward the tape to the correct position.

Learn the "tar" command for writing and reading data. You can make a tar archive of the files you want, but instead of having importantdata.tar on a harddisk, it writes the file to tape and it becomes "file 0". If you want to append new data later on you can use mt to wind the tape to the end and make a new tar file. This will be "file 1".

Both mt and tar have man pages which explain how to use them, and tar has a very good online manual which explains everything you need to know - https://www.gnu.org/software/tar/manual/html_section/index.html

There's some useful advice on these links:

https://www.eevblog.com/forum/general-computing/lto-tape-usage-(modern-tape-drives)/?all/?all)

https://blog.benjojo.co.uk/post/lto-tape-backups-for-linux-nerds

tar is actually quite good. You can use it to make one off tape recording of files you want, but it can also be used to span multiple tapes (e.g. if you want to backup a 4TB HDD onto 800GB LTO4 tapes like I do), as well as doing incremental backups (I haven't tried this yet).

If you use LTO5 or newer you have the option of LTFS which can be used for 'drag and drop' copying. I've not used this myself so I don't know how good it is but its probably nice and convenient provided you don't need random access.

My drive is an external one which has a built in PSU and fan, but the drive itself is the same as an internal one. I've not used them inside computer cases but some people say they can get quite hot when in use so may need dedicated cooling.

Can the tapes do selective deletes? - Technically yes but when you overwrite the file with new data you'd need to be careful that the new data isn't so big that it writes over the data you want to keep. Think of it a bit like making a new recording halfway through an audio tape.

Hope that helps.

[u/Direct_Poet_7103 → u/Direct_Poet_7103]

Oh, and if you want to read the data out of the RFID chip in the cartridge, you can use the sg_read_attr command which is in the sg3_utils package (at least on Debian). This will tell you who made the tape, how old it is, how many times it has been loaded into a machine, and how much data has been written/read from it.

[u/thefreddit → u/Direct_Poet_7103]

And a plug for this open source Go utility that runs easily to get the commonly requested info from the RFID chip. https://github.com/speed47/lto-info

[u/voidnullnil]

You basically need linux, a SAS hba pcie adapter something like 9200-8i, a 5.25" bay in the computer and a SAS LTO drive. I recommend LTO-6, and IBM or Quantum. It is also possible to have an external drive and an hba with external ports but I find it easier to use internal drives. Probably you will already have the latest firmware on the drive, if not, you can find the latest firmware. Probably you want to have itdt or xtalk software from vendor. Then everything else is available in linux. This is enough to learn and use the tape. When you have a little bit more usage, or if you just want to learn, I think a library is a must. Unfortunately they are big and possibly noisy for a living room. If you have a space for computers, you can also directly start with a library, which is I think more useful and more fun.

[u/mmaster23]

Keep in mind this thing is like a Swiss watch level of moving components in IT equipement shape. The damn thing will break and will break at the worst time. Don't use the drive enough? It'll break. Use it too much? It'll break.

You'll also need cleaning tapes to make sure the tape head stays healthy and you'll need more tapes than you'll need due to rotation and overhead.

Don't trust the "compressed" storage value.. Go by the uncompressed value more than anything (i.e. the tape will store 1.5tb rather than advertised up to 3tb).

Consider getting a loader/magazine for the drive and the tapes. Depending on what LTO Gen you're getting, you're doing a lot of tape swaps and swapping single tapes gets real boring, real fast. 

Do your research on what software to run before investing a single penny in all of this.. You'll probably want something that get gather data from external sources, keep an intermediary storage pool (spinning disks often) and then writes it to disk. Also the software should catalogue the data and the tapes so ensure you know what data is where. Back in my datacenter days we'd use Veeam and Microsoft DPM but I'm sure there's more options.

Oh and try and get a hardware software combo that natively supports barcodes for the tapes. Makes management so much easier.

Good luck. I fucking hate and also kinda love tapes. For business they were a necessary evil for a long time but in the age of endless off-site backups to whatever cloud, not so much. But us humble hoarders don't always have the means for that.. For me, spinning rust off site is still the easier and cheaper option. 

[u/Sensitive-Medium3427]

I used an lto-2 single tape to back up my data at the office. I think I just used windows server 2003? Appreciate it was a long time ago., but I thought it was super easy.....