Comments on: Some harsh realities http://strugglers.net/~andy/blog/2009/06/21/some-harsh-realities/ I'll get there one day. Sun, 28 Feb 2010 16:14:00 +0000 http://wordpress.org/?v=3.0-alpha hourly 1 By: Tony Whitmore http://strugglers.net/~andy/blog/2009/06/21/some-harsh-realities/comment-page-1/#comment-117976 Tony Whitmore Mon, 22 Jun 2009 12:06:30 +0000 http://strugglers.net/~andy/blog/?p=335#comment-117976 An interesting post, thanks Andy. I've often wondered (but never got down to sitting down and working out) why the storage costs on Bitfolk feel out of line with the excellent pricing of the rest of the service. Certainly when I was looking at replacing a previous dedicated "server" with a bitfolk box I worked out what it would cost to host the ~80GB of digital photos I have in various galleries and it was a bit rich for my taste. My solution was to host all the big-disk, low-risk stuff on a Dreamhost account, which can perform relatively sluggishly and occasionally drop off the 'net with no dire consequences and the more important stuff on the bitfolk box. An interesting post, thanks Andy. I’ve often wondered (but never got down to sitting down and working out) why the storage costs on Bitfolk feel out of line with the excellent pricing of the rest of the service. Certainly when I was looking at replacing a previous dedicated “server” with a bitfolk box I worked out what it would cost to host the ~80GB of digital photos I have in various galleries and it was a bit rich for my taste.

My solution was to host all the big-disk, low-risk stuff on a Dreamhost account, which can perform relatively sluggishly and occasionally drop off the ‘net with no dire consequences and the more important stuff on the bitfolk box.

]]> By: Stefano Rivera http://strugglers.net/~andy/blog/2009/06/21/some-harsh-realities/comment-page-1/#comment-117912 Stefano Rivera Sun, 21 Jun 2009 18:11:33 +0000 http://strugglers.net/~andy/blog/?p=335#comment-117912 @Andy: Aah, right, didn't realise you were doing the verify on the 3ware card too. Agreed re bit error rate, although I can't say I've personally noticed corruption caused by it, the ratio of bit error rate vs disk size is getting rather unfavourable. @Andy:

Aah, right, didn’t realise you were doing the verify on the 3ware card too.

Agreed re bit error rate, although I can’t say I’ve personally noticed corruption caused by it, the ratio of bit error rate vs disk size is getting rather unfavourable.

]]> By: Andy http://strugglers.net/~andy/blog/2009/06/21/some-harsh-realities/comment-page-1/#comment-117908 Andy Sun, 21 Jun 2009 17:37:11 +0000 http://strugglers.net/~andy/blog/?p=335#comment-117908 @Stefano, Believe me, I looked into this a lot before making the decision to spend an extra £225 per server! Already at the default speed limit it causes performance problems and doesn't complete the verify for several days. If I lowered the speed in order to not cause performance problems then I would be looking at weeks of verify. With the 3ware I verify every week and it's not noticeable. It is true of any disk (not just RAID setups) so you must verify them, otherwise you may end up in a situation where one disk dies and whilst rebuilding from the other you encounter a read error. At this point data is lost. SATA hard drives typically have an unrecoverable bit error rate of 1 in 10^14 bits read - or one error every 11TiB read - so you have to read them to find these, then you can reconstruct the data from the other disk(s) and rewrite the bad sector. If you wait until you have no other copies (such as when rebuilding after a failed disk) then you risk data loss. Newer filesystems like zfs are tackling this issue by building in a checksum. I don't think SMART checks will be this thorough, but FWIW 3ware cards do allow SMART commands through to the individual drives if you want to use SMART. @Stefano,

Believe me, I looked into this a lot before making the decision to spend an extra £225 per server!

Already at the default speed limit it causes performance problems and doesn’t complete the verify for several days. If I lowered the speed in order to not cause performance problems then I would be looking at weeks of verify. With the 3ware I verify every week and it’s not noticeable.

It is true of any disk (not just RAID setups) so you must verify them, otherwise you may end up in a situation where one disk dies and whilst rebuilding from the other you encounter a read error. At this point data is lost.

SATA hard drives typically have an unrecoverable bit error rate of 1 in 10^14 bits read – or one error every 11TiB read – so you have to read them to find these, then you can reconstruct the data from the other disk(s) and rewrite the bad sector. If you wait until you have no other copies (such as when rebuilding after a failed disk) then you risk data loss. Newer filesystems like zfs are tackling this issue by building in a checksum.

I don’t think SMART checks will be this thorough, but FWIW 3ware cards do allow SMART commands through to the individual drives if you want to use SMART.

]]> By: Stuff-as-a-service… « Alex’s Blog http://strugglers.net/~andy/blog/2009/06/21/some-harsh-realities/comment-page-1/#comment-117898 Stuff-as-a-service… « Alex’s Blog Sun, 21 Jun 2009 16:00:34 +0000 http://strugglers.net/~andy/blog/?p=335#comment-117898 [...] price of disk space - in particular, about Bitfolk, but it applies to any other service really. Andy’s take on this is really worth interesting, although I personally think he’s defending Bitfolk’s [...] [...] price of disk space – in particular, about Bitfolk, but it applies to any other service really. Andy’s take on this is really worth interesting, although I personally think he’s defending Bitfolk’s [...]

]]> By: Stefano Rivera http://strugglers.net/~andy/blog/2009/06/21/some-harsh-realities/comment-page-1/#comment-117892 Stefano Rivera Sun, 21 Jun 2009 15:08:52 +0000 http://strugglers.net/~andy/blog/?p=335#comment-117892 > Unfortunately, hard drives accumulate errors and the only way to find them is to read the disks looking for them. That's probably true of any RAID system. You might as well just disable the verification checks then. (You can also, obviously, tweak the speedlimit). I've found that if you monitor for errors with SMART (which you often can't do with hardware RAID), then you can probably do away with the verification-check. The check is a relatively recent addition, and it's never found me any discrepancies (I monitor SMART to pick up bad sectors). Disclaimer: My experiences of hardware RAID have never been positive, so I promote the use of software RAID where possible. > Unfortunately, hard drives accumulate errors and the only way to find them is to read the disks looking for them.

That’s probably true of any RAID system. You might as well just disable the verification checks then. (You can also, obviously, tweak the speedlimit).

I’ve found that if you monitor for errors with SMART (which you often can’t do with hardware RAID), then you can probably do away with the verification-check. The check is a relatively recent addition, and it’s never found me any discrepancies (I monitor SMART to pick up bad sectors).

Disclaimer: My experiences of hardware RAID have never been positive, so I promote the use of software RAID where possible.

]]>