Reply to post: Over/under provisioning

If you want a USB thumb drive wiped, try asking an arts student for help

Flocke Kroes Silver badge

Over/under provisioning

Flash devices need more internal capacity than the number printed on the outside. They need a map of logical to physical sectors that has to be stored somewhere. They need to keep track of how often each block has been erased. Some sectors do not work on new chips and some will fail while in service so there have to be spares. Finally, the ware levelling algorithm can make better choices if it has lots of unused sectors to choose from. I have found devices where the capacity of the chips add up to 50% more than the advertised capacity.

There used to be a problem with second hand chips. Old devices were recycled leading to new devices that started with a large number of bad sectors, and those that did work had already gone through a large number of erase cycles. Under provisioning is still popular. The device will work fine if you only use a quarter or perhaps a half of the nominal capacity.

A full format that writes zeroes to all the sectors not used for filesystem metadata will identify many under provisioned drives. Some of the more cunning drives will try to identify the file system, and forget the contents of unallocated sectors to increase the pool of available blocks (or to hide under provisioning).

If 90% of your drives survive a full format then you have found a supplier who works hard to detect and demand money back for under provisioned drives before they reach customers (or you picked a file system type supported by some excellent firmware).

Using dd like 1980s_coder is close to a good answer. Drive firmware is likely to avoid storing duplicate data, so half a sector might store all the zeroes, and a few more would map lots of logical sectors to that compressed sector. For a while, some of your illegal porn and bombing plans will be stored on blocks scheduled for erasure, and the firmware will get around to that in due course.

I would love to use the trim command. The latest versions of the SATA, SDHC and USB command sets all include trim or an equivalent. SATA support is common and it even works on some devices (modern Linux kernels have blacklists and whitelists). A few USB devices claim to support trim. I have yet to come across a USB enclosure that forwards trim commands to a drive.

I like to write a sequence of random numbers to a new drive, and try to read them back. That spots under provisioning. Two or three complete drive writes of random numbers will probably erase my terrorist plans. One day deleting a file could result in trim commands that are promptly and reliably obeyed, but for the next decade, the only secure erasure strategy I have real confidence in is fire.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019