Understanding SSDs vs HDDs vs USB Flash storage
Posted: Fri 03 Apr 2015, 20:33
HDD storage (and spinning disks, in general) have been used for centuries, per se. But, the landscape for data storage of all types, sequential/transactional, has seen the recent decade's entry of SSDs and USB Flash storage (this I group into a category I call solid state storage). These are devices which uses non-volatile RAM as a storage media to PCs. Even as this is new to the PC world, this type of solid state I/O storage is not new in the IT world as mainframes have had these available for over 35 years with dramatic system workload impacts where needed. The same kind of workload benefits is seen in the PC world with use of the SSDs and USB flash units.
SSDs, today, intend to be installed permanently, while USB Flash intend to be hot pluggable giving them portability. Yet both of these use non-volatile RAM with their differences, chiefly, is the technology to connect to our PCs. RAM storage will be referenced as solid state storage in this post. S
Edited: Some late news on SSDs are discussed/shared here in this Puppy forum.
Using this kind of storage devices, just as was the case 3 decades ago, the same device data storage problems exist in today's PC arena. Device utilization and I/O channel utilization have an increasing impact, over time, as individual SSD/USB flash device use increases in the PC's operations. To address device problems, the units have intelligence on the units to manage these traditional problems. To do so, units require space on the unit to be free. This available space's use is called "over-provisioning". In some early PC drives this is made available by the administrator at installation-setup time, manually. And, in the past, many users have unwittingly allocated ALL solid state space to writable partitions without understanding their requirement to leave free unallocated space for over-provisioning. Over-provisioning is NOT required for HDDs. Yet,the over-provisioning aspect of solid state storage is an important function in data management and for maintaining data integrity. Over-provisioning is a reserve of free AND unallocated AND unpartitioned space which is available so that there are replacement storage blocks available when actual data storage blocks on SSD/USB become defective. And, some solid state storage units use this space for "TRIM and Garbage Collection" which is done in a different manner than is seen on a HDD. In a HDD, when a data block is freed, it is marked and can directly be reused. NOT SO in the SSD world, as, the data block must be emptied clean when it becomes available BEFORE any new data can be written to the free block. This is NOT a problem as is done as a background operation on the solid state storage when no I/O operations are active.
So question: "What do you think happens when you have lots and lots of R/W operations occurring at a busy time where the SSD has run short on actual clean blocks to write knowing that you have blocks on the device which are free but have not been cleaned? Yes, you guess correct; you must, either, stop all I/O activity to clean blocks before continuing or you will employ some kind of method to clean then write, then clean more then write, then ... one block at a time. As you can see, THIS is detrimental to the solid state devices performance and your system is sitting having to wait for this.
Now that we understand why this free unallocated space is needed, how much should be free for its use? I have seen recommendations over the 3+ decades from 15% to 40% of free space, when manually done. These recommendations vary. In the case of newer modern SSDs/USBs, the manufacturer does this for you by leaving as much as 10% untouchable or unseen by the OS for the device's management needs.
For optimum reliability (and to a lesser extent, performance) SSDs require some free space, which, without going into too much detail, they use a writing method for spreading data around the drive to avoid constantly writing to the same place (which wears them out). Most everyone has heard that you do NOT want to write (or read either) too many times to one place on these devices. Just as was the case decades ago, the same is true with these types of storage systems in today's world. The solid state storage industry builds in a feature called "load-leveling" which works as described. That is, it spread data all over the device so as not to prematurely wear out individual blocks by attempting to even disperse data blocks use. When the devices are new, no blocks were ever written to, so data file(s) is usually written contiguously to SSD/USB blocks. As time goes on and every block has been written to, at least once, load-leveling is at work to spread new data around with fragmenting of files being a normal behavior of the device as load leveling operates. Normally, for most every workload using solid state devices, this does NOT cause system/applications problems. But, for some workloads, this is detrimental even though these devices are "direct access" devices going directly to its RAM for the actual block of data needed when reading or writing (this can be seen where some system elements may require data to be contiguous during boot use, etc).
HDDs have traditionally had this problem of fragmentation, too. But unlike solid state storage, they do NOT have a load-leveler at work and in many, many more cases, data will NOT be fragmented excepting due to some PC I/O subsystem decisions. Fragmentation of files on a HDD means that the HDD must use its mechanical means to spin and move arms to find data locations for files which became fragmented across the drive. This can and does have significant impact on workloads accessing HDD data as timings are affected waiting for data to be plucked and presented from the spinning platters. SSDs are memory devices, in normal use (not the special cases mentioned earlier) is NOT impacted as it finds its data using a direct location means where the time is always the same no matter if its the 1st memory block on the unit or the last. By comparison, a single data block's access via SSD might be 0.1ms while the same data reference on HDD may be 15ms; an ENORMOUS difference.
Thus, these are items you MUST be marginally aware of as you expect your solid state storage to be used in a fashion where you understand why your system may/may-not slow at times in the use due to the unit's internal management; and, how the device actually operates with your data usage. Solid state storage devices have brought us much beneficial use. Along with it, we have a responsibility to understand its behavior.
Most everyone of us have been around HDDs for so long that we understand HDDs.
Hope this helps the few in better understanding this PC storage.
Contribute to this thread as you see appropriate. All contributions for use and understanding is welcomed.
SSDs, today, intend to be installed permanently, while USB Flash intend to be hot pluggable giving them portability. Yet both of these use non-volatile RAM with their differences, chiefly, is the technology to connect to our PCs. RAM storage will be referenced as solid state storage in this post. S
Edited: Some late news on SSDs are discussed/shared here in this Puppy forum.
Using this kind of storage devices, just as was the case 3 decades ago, the same device data storage problems exist in today's PC arena. Device utilization and I/O channel utilization have an increasing impact, over time, as individual SSD/USB flash device use increases in the PC's operations. To address device problems, the units have intelligence on the units to manage these traditional problems. To do so, units require space on the unit to be free. This available space's use is called "over-provisioning". In some early PC drives this is made available by the administrator at installation-setup time, manually. And, in the past, many users have unwittingly allocated ALL solid state space to writable partitions without understanding their requirement to leave free unallocated space for over-provisioning. Over-provisioning is NOT required for HDDs. Yet,the over-provisioning aspect of solid state storage is an important function in data management and for maintaining data integrity. Over-provisioning is a reserve of free AND unallocated AND unpartitioned space which is available so that there are replacement storage blocks available when actual data storage blocks on SSD/USB become defective. And, some solid state storage units use this space for "TRIM and Garbage Collection" which is done in a different manner than is seen on a HDD. In a HDD, when a data block is freed, it is marked and can directly be reused. NOT SO in the SSD world, as, the data block must be emptied clean when it becomes available BEFORE any new data can be written to the free block. This is NOT a problem as is done as a background operation on the solid state storage when no I/O operations are active.
So question: "What do you think happens when you have lots and lots of R/W operations occurring at a busy time where the SSD has run short on actual clean blocks to write knowing that you have blocks on the device which are free but have not been cleaned? Yes, you guess correct; you must, either, stop all I/O activity to clean blocks before continuing or you will employ some kind of method to clean then write, then clean more then write, then ... one block at a time. As you can see, THIS is detrimental to the solid state devices performance and your system is sitting having to wait for this.
Now that we understand why this free unallocated space is needed, how much should be free for its use? I have seen recommendations over the 3+ decades from 15% to 40% of free space, when manually done. These recommendations vary. In the case of newer modern SSDs/USBs, the manufacturer does this for you by leaving as much as 10% untouchable or unseen by the OS for the device's management needs.
For optimum reliability (and to a lesser extent, performance) SSDs require some free space, which, without going into too much detail, they use a writing method for spreading data around the drive to avoid constantly writing to the same place (which wears them out). Most everyone has heard that you do NOT want to write (or read either) too many times to one place on these devices. Just as was the case decades ago, the same is true with these types of storage systems in today's world. The solid state storage industry builds in a feature called "load-leveling" which works as described. That is, it spread data all over the device so as not to prematurely wear out individual blocks by attempting to even disperse data blocks use. When the devices are new, no blocks were ever written to, so data file(s) is usually written contiguously to SSD/USB blocks. As time goes on and every block has been written to, at least once, load-leveling is at work to spread new data around with fragmenting of files being a normal behavior of the device as load leveling operates. Normally, for most every workload using solid state devices, this does NOT cause system/applications problems. But, for some workloads, this is detrimental even though these devices are "direct access" devices going directly to its RAM for the actual block of data needed when reading or writing (this can be seen where some system elements may require data to be contiguous during boot use, etc).
HDDs have traditionally had this problem of fragmentation, too. But unlike solid state storage, they do NOT have a load-leveler at work and in many, many more cases, data will NOT be fragmented excepting due to some PC I/O subsystem decisions. Fragmentation of files on a HDD means that the HDD must use its mechanical means to spin and move arms to find data locations for files which became fragmented across the drive. This can and does have significant impact on workloads accessing HDD data as timings are affected waiting for data to be plucked and presented from the spinning platters. SSDs are memory devices, in normal use (not the special cases mentioned earlier) is NOT impacted as it finds its data using a direct location means where the time is always the same no matter if its the 1st memory block on the unit or the last. By comparison, a single data block's access via SSD might be 0.1ms while the same data reference on HDD may be 15ms; an ENORMOUS difference.
Thus, these are items you MUST be marginally aware of as you expect your solid state storage to be used in a fashion where you understand why your system may/may-not slow at times in the use due to the unit's internal management; and, how the device actually operates with your data usage. Solid state storage devices have brought us much beneficial use. Along with it, we have a responsibility to understand its behavior.
Most everyone of us have been around HDDs for so long that we understand HDDs.
Hope this helps the few in better understanding this PC storage.
Contribute to this thread as you see appropriate. All contributions for use and understanding is welcomed.