Gearing up for a huge NAS purchase - questions about use

General Off-Topic
#1

So I could ask this on a regular plane jane PC-esque forum, but since its use will primarily be for emulation (and a minor hunk for plex) I figure this will be easiest to figure out what needs to happen.

A bit of info on my current set up. I have 50 terabytes of storage at current spread across 9 drives. 3 of those 9 are plex and external, the other 6 are internal. 0 redundancy so if a drive goes - im ass-out.

This is half the reason I need a NAS. I want to do something like a raid 6, which will allow me a bit of protection against failure. This equals needing a few more drives. Add in the fact that for the money i’ll be spending, I also need to actually UPGRADE my capacity, which equals a few more drives yet.

I’m looking at doing something like 10x10tb drives, lest 2 for parity.

All that stuff is the easy stuff though. What concerns me is how it interfaces with the PC.

As mentioned, I currently have 9 drives. All pathed out. Plex has library paths to each of the 3 drives its assigned (which is a very small fraction of my concern and will be fixable or even rebuildable much easier than what I have going on with retroarch).

Retroarch on the otherhand (not to mention several standalone emulators for which retroarch has no equivalent core) has playlists which point to various drives, symlinks for art which point to their own drive, and even steam has command line shortcuts that rely on retroarch being in a certain place.

All this was curated and set up over an embarassing amount of man hours that cost me a large amount of family time (and game time!).

Hindsight and all that, I should have went with a NAS years ago. They were less prevalent though and i never realized i’d become so compelled to create a veritably complete retro-gaming collection.

So the easy out is to simply buy 9 new/larger drives, transfer 1:1, assign the same relative drive letters to each, and be done. The old drives would then be backups of a sort.

Still considering all the other perks that having a NAS brings, I want to give it a go if its reasonble to set up.

So ultimately the TLDR version is, would a NAS require me to redo every path/playlist/shortcut/symlink for every single thing?

ive briefly heard of/read about something along the lines of “mapping it as a system drive” or some such, which in my head ideally would work like this.

-create a bunch of partitions on the NAS each holding the data from each of my drives. Assign each partition as a drive to windows with the same letter as the current drive.

leave happy with everything functioning as it always has.

Prooooobably unfortunately not how it goes in practice. Since im old and married (aka friendless), and as such have nobody with a NAS i can go visit and poke around with - I’m left with the next best thing. The internet! I have to rely on someone out there that has seen and used a NAS to set me right :slight_smile:

#2

For the solution, look at what best fits your needs. Raid is faster, I believe had the same features as a standard drive, and provides parity but is inflexible if you need to expand your system, requires quite a bit of work to set up and opt out, and you lose everything if you lose more drives than you have parity for. If those risks are too high, I’d look at other solutions like unraid, snapshoting, and drive pooling which will have their own benefits and problems.

As for the links, you’ll need to reset them up and test but there are a few ways of doing it. The right way is to go through and fix everything as possible, making sure it works (you can probably mass find and replace the paths in your playlists at least. The quick way is, assuming you keep the same folder structure, to try something like creating sym links from your old drive letter to the new drive letter. Or You might be able to mount the same physical drive as several logical drives.

I have an 8 drive system set up through drive bender (a pooling program) and then two drives of parity done through snapraid. The upside is I have a single, easily expandable and reversae logical drive but it can be slow、drive bender doesn’t support sym links, and I have to remember to backup. I’m not sure what I would choose now, but back in 2013 this was a pretty good option.

#3

Thanks for the reply.

Its very likely ill end up going with one of those prebuilt synology or qnap jobbies and filling it with some ironwolfs.

In all honesty I havent gotten to that point before I was able to get some idea of what I was getting into as far interfacing goes.

You are correct, the playlists should be, by and large a copy/paste notepad++ job. Its the recreation of all my steam shortcuts, my windows games shortcuts, my vm’s and damn near everything emulation related (scrapers etc) that all point somewhere that will no longer be valid.

From what it sounds like, theres not really gonna be any clear cut way around it. While it will/would be a ton of work - perhaps its best in the long run.

I appreciate the input, especially from someone with hands on experience in exactly this realm.

hows performance relative to a local disk? Is it roughly the same? I hear a lot of talk about latency with NAS setups but I figure im gonna be going from 5400rpm archive drives to 7200rpm drives in a raid, so hoping to at least break even.

As for expansion I believe even with raid its possible, your just stuck changing out 1 disk at a time and its slow as can be while it rebuilds. Especially with volume of data.

#4

Yeah, if you are going to go the long route I’d recommend spending some time thinking about how you should arrange your drives, folders, etc so that should you change your storage solution in the future you don’t have to redo all of your links etc again.

For performance, I haven’t really noticed any issues while gaming but Drive Bender does cause a bit of slowdown in File Explorer. It can sometimes freeze when working with a folder with lots of files, moving/copying lots of files, and in the past would sometimes slow the system down during indexing though that’s fixed now. I think those problems are pretty specific to software-based pooling solutions so I doubt you’d see any issues with a raid etc.

#5

Hmmm.

Who woulda thought it was such a big choice. Ultimately a NAS solution would potentially allow me 1 folder for roms, movies, tv shows, etc - versus several spread across several drives and mapped differently etc.

For all intents and purpose it would operate like a big disk should i choose.

The flipside is, if i put the money, time, and work into making that “conversion” we’ll call it… Then im now locked into that for basically ever. I imagine re partitioning that out to individual disks would be an even larger PITA.

Not to mention, when it comes time to change NAS units, I cant even guess at what downtime would be with 50-100TB of data. I “think” it can be reasonable should u stay with the same filesystem/raidtype/software/nas brand, but any time you were forced to legitimately migrate - scary thought.

What would you do? Do you think the merits of what a NAS offers are worth buying into that as your primary storage solution for…well good?

#6

I haven’t done the research on NAS for a while so I can’t speak to that specifically but I can break it down a little.

I think there’s a big benefit of converting everything over to a single disk (either through raid or pooling or whatever) as it makes finding files a lot easier and you don’t have to track the fill status of like 10 disks whenever you add a new game.

I also think there’s a big benefit to redoing your file structure and sym links to try to minimize the effect of future changes. I would try to move everything that HAS to be in a specific place to somewhere static like C: and then store the locations of the rest of your files in that location. That way, should you change your solution in the future you just need to go to that folder on C and update the links

I am simultaneously terrified and intrigued by RAID and would probably look for other options that would let me stripe and offer parity but don’t lock you into disk size limits and massive rebuilds every time you swap disks until I gave up and chose something but no idea what.

#7

well ideally, you’d never have to rebuild at least lol. I’m so far so good on my 9 disk pileup, knock on wood.

I do have them set to spin down and some dont get used for days, so im sure that contributes to my “luck” and the fact that I upgrade them every couple years at worst. Usually sooner if i run out of storage space, and if not, when i start feeling scared.

also infrequently peek at the smart data etc.

youve given me a substantial amount to think about :slight_smile:

#8

Look into buying/building something and installing Unraid on it. I use it for everything now and love it. Although it’s not as easy to set up as a standard NAS.

#9

yea, just last night I was doing some digging as the price of a pre-built NAS unit is pretty up-there relative to what you actually get. Not to mention your largely locked into that ecosystem at that point.

Im not averse to a pre-built, while I have no experience with a nas and VERY little with linux, I have been building and working with pc’s for 20+ years.

The first hurdle I found is finding a chassis that’ll take 12 3.5" disks lol. Obviously dont want rack mount, and for at least a bit of future proofing + failsafe’ing I’m going to need 12 bays.

#10

I was in the same boat as you with very little Linux knowledge and no NAS knowledge barring a short time spent with Synology but I think you’ll be fine considering you’ve been building computers for so long. There are lots of videos and lots of helpful people in the Unraid community.

I struggle to find a lot of chassis’ that have 8 bays let alone 12. A quick Google however came up with the Lian-Li PC V2000B PLUS. Now how good a chassis it is I don’t know but it has 12 bays and doesn’t seem to be stupidly expensive.

Hit me up in a PM if you want to talk more about Unraid, it’s a really interesting piece of software and I love talking about what it can do.

#11

ah, i hadnt come across that one. Havent used an lian-li case in yeaaaaaaaars. PC-71B i think for a build back in the early 2000s maybe even late 90s. dont recall :stuck_out_tongue:

Still - I hadnt came across that one. Perhaps it was due to the form factor I was looking for. Something less conspicuous and less pc-esque. Still shakey on what differentiates a NAS from a file server, but ultimately want something that can be simple in operation and appearance most of the time. Gonna be a big financial investment and a big initial set up, so beyond that point Id love mostly hands off.

Either way, this post has brought me even more to think about - so when I do get to such a place where ive decided on a DIY relative to an AIO, if thats the direction I head - I’ll definitely take you up on discussing Unraid.

#12

Yeah, looks like it’s an old chassis though looking again (USB 2.0 only front panel connectors) so that may or may not put you off.

You might be better with FreeNAS now I think about it if you’re looking more for a file server and will use another device for your Plex server, etc.

I just loved the idea of one server which I can run VM’s on, use it as a file server, Plex/Emby server, download box, router, etc, etc but not everyone wants all that in one box and I understand that as well :slight_smile:

#13

in all honesty, it will be first and foremost a means to consolidate paths, and have a bit of reassurance regarding drive failures. (likely raid 6).

It will also host plex media and I’m still torn on whether or not it should run plex server or leave it to my pc. I also like the idea of not having to have my pc on 24/7 like I do now. The tradeoff of course would be needing something powerful enough to transcode.

It will also hold my VM’s but they’d be ran on the pc, so apart from read speeds I dont know how performant it would need to be in that regard.

I did hear people repeating FreeNAS quite often, and theres plenty of debate on the whole AIO vs DIY thing.

Add the aforementioned to my perfectionist/ocd’ism (plus the fact its a HUGE investment for me) and this ones gonna have to stir n simmer for quite some time. Might as well as I imagine the initial set up and transfer of data is gonna take literally weeks. (which is another factor in my choice of which route to take as if i make the leap, im definitely not going back).

#14

One advantage (regarding emulation) of going for a NAS instead of multiple DASes is that you can share the art/rom/saves collection with lots of computers at the same time.

But that of course would probably require to set all the folders again. But if go for a DIY NAS route, it probably may be the last time you set the folders.

I had at a time 3 desktops and a Lakka@RasPi connected to the same Rom/saves/thumbnails share on a NAS.

.

Using a basic RAID 1 setup, all connected via gigabit ethernet, I got around 80~100MB/s of sequential read/write speeds. But of course with more clients connected, the speed dropped. And wireless clients got nowhere near that speed.

#15

Hi, thanks for the reply.

Im still toiling over this. For what I need/futureproofing etc, the cost is going to be VERY large. At least 4k (large for me at least lol).

Point being, I have to make the right call. The other option is of course buying bigger singular drives and using them as network shares as I currently do.

I can live with the paths updating. Like you said, it will likely be the last time i’ll ever need to.

Whats more concerning now is performance, the future, and degrees of failure.

Let me start with performance.

I currently use large 5400 rpm archive drives (SMR). Theyre slow but it hasnt effected me “too” much best I can tell. Unfortunately i have 0 to compare it to other than my m.2 nvme system ssd so yea, not apples to apples lol.

If i used the NAS, everything would be going over to it, including the emulators/frontends themselves which are currently running off of said SMR drives. (nothing emulation related is on my ssd, space premium etc etc).

I hear a lot of people speak about latency in regard to a NAS and im wondering if id see the same, better, or worse performance running games from a NAS using 7200rpm quality disks in a raid 6 over gigabit? (relative to 5400rpm local archive drives connected via sata)

If performance would be equivalent or better, that leads me to considering the next point - the future.

Right now, every few years I simply buy a new drive(s) copy the data over, assign the same drive letter - and carry on as normal. With a NAS that complicates things dramatically. When it comes time to upgrade the drives, how does that even work? Its my understanding you can just pull/rebuild one at a time - but in case of a failure, ive heard a single 10TB disk can take days to rebuild and puts tremendous strain on the remaining drives. So going through that 10-12 times for all the disks sounds insane.

Or lets say we’re not upgrading drives, but upgrading NAS units - from what I understand, some support “direct migration” which is basically unplugging your disks, popping em into the new one, and carrying on as usual. Thats great. Unfortunately I think that leaves you entirely locked into a specific brands NAS and the hope each successive purchase also supports direct migration. Otherwise your left buying a second 100TB of storage to transfer your data off and start fresh. Also insane.

This leads me to my next and final point. Failure.

Cloud backup isnt an option. At current I have no redundancy whatsoever. If a drive fails, I lose everything on that drive. I keep them cool, I allow them to spin down (sometimes for days depending on which drive) I check the smart data, and I change them out every few years. So far, ive never seen a failure. Ive been fortunate.

A NAS would at least provide me protection against (in my intended use-case) the failure of 2 drives simultaneously. One would think thats Exceptionally unlikely. I know I did. But then when I read about the fact that A) your buying all your drives at the same time and B) when one fails, your stuck rebuilding which puts strain on the remaining - I begin to wonder.

Or consider if the NAS itself dies. Then we go back to my 2nd point, you have to find a way to get that into a new NAS. Not to mention the cost.

So im left with at current, no protection against failure other than diligence and a little luck. A failure will lose 10TB of data give or take, but only cost a few hundred bucks to replace.

If i go NAS, i have more protection against a singular failure, but a NAS failure would present much bigger problems at a WAY more substantial cost and compromise or potentially compromise ALL the data.

In the end it leaves me really toiling over the pros and cons. Is a NAS going to be faster than my current solution? Is it going to be the safer route? Will the conveniences be offset by a loss of longevity/higher chance of failure? As i mentioned, I have 10 disks at current. Theyre all slower drives that spin down. Saves lots of heat and power, and I imagine for drives that are mostly written to once, and just used - theyd last a LONG time.

Now 7200rpm drives inside a NAS, all spinning simultaneously for 18 hours a day, every day. I wonder if it would be just as safe, with better longevity and be more cost effective to keep at it the way ive been going.

Simply opening up the drives as network shares.

My main desktop is basically my NAS. Its obviously no good if i want to access stuff remotely, but for other pc’s within the house its worked thus far. At my age with kids, I dont see myself needing to get at this stuff from outside the house. Would it be a nice convenience to have? Sure - but veeeery far from a selling point.

I dont even have multiple people using media simultaneously. Maybe the wife or kids using plex off of the plex drives, and me gaming off the gaming drives.

I just have an emulation hobby thats spiraled out of control so Im wondering if going through this once is the better choice for the longhaul. To put everything in one place and have access to it from other devices a bit cleaner/easier.

So the TLDR version is considering all of the above, what should I do?

I’ll be upgrading all my drives soon, its that time. Gonna double my storage capacity and its gonna cost me at least 2500 going at it with local disks. A NAS will add substantially to that for a 12 bay solution.

Will the added cost provide enough relative to the drawbacks? Or maybe some of my concerns are moot? I dont know as ive never owned or even saw another nas in person/setup.

sorry for the novel but these are all questions ive been toiling over that ive yet to put on paper in any one single place. Your post was the catalyst :stuck_out_tongue:

#16

As you pointed before, you can choose a lower power cpu for the NAS - if going for the DIY route - that would probably save power in the long run, as you could leave your main desktop off.

If you need to transcode for Plex and wanna go DIY, you would need to find NAS distros that will convert the files for you.

I personally chose a prebuilt Qnap (have two, one at my parent/brother´s home, another at my place) but in case the Qnap dies, I will be stuck buying another Qnap case probably.

Regarding latency, it is higher than a local drive indeed. But not in a way that hampers my current usage (media share, rom/saves/thumbs share), which is not really latency-sensitive. One thing I noticed tho was that wireless clients had some trouble accessing saves at the NAS - but there are lots of interference around that specific desktop, the machines connected with cable didn´t present problems…

Latency may be a problem with VMs, tho. I was going to use mine for that, but didn´t even choose between VMWare or VirtualBox…

I probably would consider building a DIY, the points of failure are probably lower than using all the drives directly in your main PC. And while rebuilding may indeed take a while, that is the price to pay for redundancy. (BTW, I replicated my older NAS to the new one and it took me around a day to mirror a 4TB drive)…

An interesting middle-ground would be pre-built FreeNAS systems: https://www.freenas.org/freenas-mini/