Adding new disk drive

[davidedric]

Hi,

I use Lightroom 5 for the great majority of my pp.

Currently, I have an ssd hosting os, programs, Lightroom catalogue, and image files. This was only ever planned to be a temporary measure.

I plan to leave the os, programs and catalogue on the ssd, and add 1tb disk for the image files (there is a multi stage back up approach behind this)

Question. Completely disregarding the cost, would you choose an ssd or a conventional hd. I am thinking of reliability, and any performance advantage.

Dave

[Simon Garrett]

Generally SSD for random access speed and conventional HD for reliability. 1TB SSDs get a bit expensive, too.

I put catalogue, previews and camera raw cache on an SSD, image files on a conventional HD.

[Manfred M]

HDD for sure. It is safe, proven technology and frankly you are not going to need a lot of speed if you are looking purely at a storage drive. The only thing I would add (if your machine can be configured that was) is to put in two drives in RAID configuration to add that layer of redundancy.

[DanK]

I use HDD for photos and an SSD for the program, with an external HDD (instead of RAID) and Crashplan for backups of photos. There isn't much to be gained by having the photos on an SSD, and SSDs are far more expensive.

[Colin Southern]

Generally SSD for random access speed and conventional HD for reliability. 1TB SSDs get a bit expensive, too.

I put catalogue, previews and camera raw cache on an SSD, image files on a conventional HD.

I can tell you now that an SSD is orders of magnitude more reliable that conventional HDDs; the data shows that, and I've found that to be the case in practice too. The ONLY downside I've found with SSDs is the cost per GB.

Interestingly, hybrid drives are appearing now - HDD foundation with around 60GB of SSD caching; great for speed and good price/performance compromise.

I'm betting that in around 5 years conventional hard drives will be a thing of the past; I certainly don't use them for customers anymore.

[Simon Garrett]

I can tell you now that an SSD is orders of magnitude more reliable that conventional HDDs; the data shows that, and I've found that to be the case in practice too.

That's interesting. Can you post any references to that?

When I was buying a second SSD a few months ago, the test reports I read showed lifetimes still quite a lot shorter than HDs in terms of number of reads and especially writes. I can't remember where I was looking at the time and I'm going out now, but will try to find the reports again when I'm back tomorrow. But I'd be interested in any contrary test results.

[Colin Southern]

That's interesting. Can you post any references to that?

When I was buying a second SSD a few months ago, the test reports I read showed lifetimes still quite a lot shorter than HDs in terms of number of reads and especially writes. I can't remember where I was looking at the time and I'm going out now, but will try to find the reports again when I'm back tomorrow. But I'd be interested in any contrary test results.

Hang on a moment first - are you talking reliability or longevity?

[dabhand]

Hi,

I use Lightroom 5 for the great majority of my pp.

Currently, I have an ssd hosting os, programs, Lightroom catalogue, and image files. This was only ever planned to be a temporary measure.

I plan to leave the os, programs and catalogue on the ssd, and add 1tb disk for the image files (there is a multi stage back up approach behind this)

Question. Completely disregarding the cost, would you choose an ssd or a conventional hd. I am thinking of reliability, and any performance advantage.

Dave

Dave - the setup I use is O/S, Programs and catalogues on one SSD; caches, scratches etc on another SSD and images on a high speed HDD. The SSDs are backed up to the HDD and then they are then all backed up to a NAS drive set to raid1 using a simple timed batch program. Major belt and braces I know, but as it essentially costs nothing and all the backups are automated, it's no big hassel.

steve

steve

[davidedric]

Hi Steve,

Thanks for the response. Do you have the programs and catalogues separated from the caches etc for performance reasons, or for size, or for something I can't think of?

Dave

[dabhand]

Dave - essentially I do it that way to avoid putting all important eggs being in one basket.

Yes, in theory it will help performance, though I've no comparison of the real and theoretical measures and if they are actually significant (what's a second or two when I'm sipping a coffee), so I doubt there's much difference in speed if the caches etc were on the same drive as the programs - but without backups, recovery from failure could be a royal pain.

steve

[davidedric]

Thanks, guys. As well as the good information, it makes me think about what I am really trying to achieve. I think I know what I am going to do next.

For completeness, my current set up is an

- ssd with os, programs, Lightroom catalogue and Lightroom image files
- twin 2tb hdd's in a RAID 1 configuration. Holding all my other data, sooc images files, and daily Lightroom catalogue back-ups
- an external 2tb usb3 connected hdd, to which I backup manually key data at important times. For example, after a major shoot, images imported in Lightroom and culled, I will copy both the image files and catalogue to the external hdd. Gives me a kind of checkpoint.

I have a (possible irrational) fear of software managed incremental back-ups, because you really don't know how recoverable they are until you come to need them. It's why I prefer full copies.

So what to do, given that I need to shift the images off the ssd. It now seems that the obvious thing to do in the first instance is to move them to the mirrored disks, where I do have plenty of space. That way I can see whether I notice any performance degradation. It does reduce the redundancy in the system from three copies of images (four if you count the hdd's as two) to two (ditto), but I think I can live with that. Then have yet another re-think of the whole back-up thing.

Thanks again,

Dave

[IzzieK]

Dave - essentially I do it that way to avoid putting all important eggs being in one basket.

Yes, in theory it will help performance, though I've no comparison of the real and theoretical measures and if they are actually significant (what's a second or two when I'm sipping a coffee), so I doubt there's much difference in speed if the caches etc were on the same drive as the programs - but without backups, recovery from failure could be a royal pain.

steve

Steve by experience of a long time ago, assigning your scratch disk to another HDD helps with performance. I even went to the extent of assigning my scratch across several disk drives. These are assign-ables so in clean-ups you can physically erase the files after you finalized your editing and shut off your program, in my case, Photoshop.

[dabhand]

Steve by experience of a long time ago, assigning your scratch disk to another HDD helps with performance. I even went to the extent of assigning my scratch across several disk drives. These are assign-ables so in clean-ups you can physically erase the files after you finalized your editing and shut off your program, in my case, Photoshop.

Izzie - yes in the past I would try every trick in the book to optimise throughput in both work and home systems, but paradoxically, now I've retired and don't require immediacy, with an i7 processor, 16GB memory and SSDs, the system is finely tuned and fully capable of speedy work - unlike its operator !!

steve

[davidedric]

Izzie - yes in the past I would try every trick in the book to optimise throughput in both work and home systems, but paradoxically, now I've retired and don't require immediacy, with an i7 processor, 16GB memory and SSDs, the system is finely tuned and fully capable of speedy work - unlike its operator !!

You've been reading my mail

[Simon Garrett]

Hang on a moment first - are you talking reliability or longevity?

The two are not unrelated. Do you have any sources?

I haven't had much time to search yet. I can't remember where I got my earlier information, and most of what I've found tonight is apocryphal, not citing sources. Interestingly, I have found quite a few recent articles (last 12 months) saying that SSD reliability is greater that that of HDs. However, as far as I can tell most of those come from a single source, as they quote the same figure: "SSD Annual Failure Rates Around 1.5%, HDDs About 5%". That figure comes from an IHS report by Fang Zhang ("Storage Space Market Brief - Issue 2 - 2013", 4th Jan 2013), but I can't get a copy (it's available to subscribers only) and I don't know how she measured failure rates, or who sponsored the report, or even whether it was original research. The only other source I could identify was an Intel report which I also couldn't get, but references to it doubted its impartiality, let's say.

If you have any information giving test methodology and detailed results of failure by time and by usage, I'd be very insterested. I mean: in a way that allows a useful comparison between SSDs and HDs. There are any number of reports of failure characteristics of individual gates in NAND Flash memory, or seek errors in HDs, or the like - but not useful for comparisons.

[Colin Southern]

The two are not unrelated. Do you have any sources?

I did some further Googling last night when you expressed a further interest, but to be honest ...

1. It's a can of worms because some brands like Intel are enjoying failure rates (0.3%) that are far lower than other brands (like OCZ).

2. It's a moving target anyway (they're all improving)

3. HDDs are a moving target too

4. Statistics really don't mean a lot for any individuals purchase (ie you really don't care if nobody else has had a problem if you have a problem)

From the coal face, I can say that I've put dozens and dozens of the Intel ones into systems without a single failure to date (probably coming up on a couple of years now). In contrast, I see quite a few HDD failures each year -- especially with laptop HDDs (SSDs for laptops are practically a no-brainer these days).

Reliability is kinda saying "how long before it breaks down" whereas longevity is more "how long before it wears out"; the first being a "random" event whereas the 2nd is expected at a certain time. For what it's worth, I've been running the Intel SSD toolbox on a number of drives - including some that have been going 24 x 7 for over a year, and they're still showing their estimated life remaining as being 100%. People get worried about them wearing out, but all I can say in practice is that it's not been an issue at all.

Hope this helps.

[Ady]

Regarding SSD longevity: the thing to look for is the quoted TBW (total bytes written), it's a number most manufacturers hide away pretty well but given that it forms part of the warranty agreement from most manufacturers (drives significantly over their TBW may be refused warranty replacement) it's something they should publish. Sometimes they give the TBW as an equivalence of xGB per day for period y.

For example Samsung quote a TBW of '72TB or 40GB per day for 5 years' for their 840 EVO drives (of which I have a bunch), these are cheaper desktop SSDs so don't represent the top end for either longevity or performance but it gives an idea of what you might expect.

However most manufacturers are quite conservative and many drives will exceed their TBW by a significant margin.

So when evaluating drives look at performance, TBW, MTBF and published failure rates, though you may have to spend some time digging around the internet for some of this info, alternatively try contacting the manufacturers.

Bottom line is that most decent (cherry picked/enterprise) spinning disks will last longer than most SSDs when measured by bytes written.

When it comes to reliability (often measured by failure rates within warranty, and TBW for SSDs) there appears to be a slight advantage to SSDs.

Unfortunately its not always easy to come by all the relevant data to make full and meaningful comparisons and if it's something you are particularly concerned about you may need to do quite a bit of research if you want definitive numbers to base your choices on.

When it comes to all component choices I work it down to a small number of options (based on experience and anecdotal evidence) then pursue the data for those few, otherwise by the time you have all the info to make a fully informed decision the components you were looking at will have been superseded.

I used to spec/test/buy for a desktop and server build floor and unless the industry has completely gone to pot in recent years its worth talking to your local quality system builder. You won't get the cheapest prices but (if they are good) they will have done all the research not only on individual components but also on matching the different components in a system for compatibility, reliability and performance.

Cheers,
A

[Ady]

If you have any information giving test methodology and detailed results of failure by time and by usage...

Results may be difficult to get from manufacturers but for methodology try JEDEC, you will need to register to download documents but it's free.

Cheers,
A

[Simon Garrett]

Thanks Ady and Colin for the information, much appreciated.

Obviously a direct comparison of SSD and HD failures is tricky, given the different failure mechanisms.

For example Samsung quote a TBW of '72TB or 40GB per day for 5 years' for their 840 EVO drives (of which I have a bunch), these are cheaper desktop SSDs so don't represent the top end for either longevity or performance but it gives an idea of what you might expect.

Thanks, that set me searching for more information. As I understand it, lifetime is ultimately dictated by program/erase (P/E) cycles per bit, so presumably the TBW figure will depend on drive size. I see figures quoted of around 1,000 P/E cycles for TLC technology (e.g. Samsung 840) and 3,000 for MLC technology (e.g. Samsung 840 EVO and PRO). Starting from those figures, calculations I've found in reviews come out at around 5 years at 40GB/day for Samsung 840 TLC, and 2-3 times higher for Samsung 840 MLC. The calculations depend on overprovisioning, estimates of write-amplification and so on. Whatever, it comes out at a long time for normal desktop use (probably less than 20GB/day).

But it does also sound as though early-year failure rates per year are now better for SSDs than HDs.

With fairly limited experience of running servers, what we found in the past was that HD lifetime was largely dictated by total running time. As a result, with a batch of servers (and hence HDs) bought at the same time, the HDs would all start failing at the same time. An important lesson: even if you're using RAID, when one drive fails, replace it immediately, as others will fail soon. I would imagine the same would apply to SSDs (though as a result of P/E cycles rather than time).

[Ady]

As I understand it, lifetime is ultimately dictated by program/erase (P/E) cycles per bit, so presumably the TBW figure will depend on drive size.

Hi Simon,

yes you'd think so, but for reasons unknown to me (and to be honest I'm struggling to find the desire to find out) Samsung quote the same TBW (72TB) for all their 840 EVO drives from 120GB to 1TB.

I have a bit of experience with large drive arrays in various storage environments and while it's always tempting to fill your enclosures from the outset to cater for the growth in storage requirements it can be very worthwhile to only buy disks as they are needed so you don't end up with dozens of disks all starting to fail within a short window. And of course the more disks you have the greater the likelihood of seeing failures, even with the reliability of modern disks if you've got a rack full of storage you are pretty much guaranteed a failure every few weeks or even days, hot spares are a life saver.

Cheers,
A