Re: [AMIA-L] Preserving digital video on Removable Hard Disk Drives- warni ng ;-)

[Zwaneveld Eddy <e.ha.zwaneveld@xxxxxx>]

Greetings,

We are encountering some enthusiastic 'truths' from a couple of recent
converts to the new religion of the worshippers of 'temporary life'. Who
preach Hard Disk Drive heaven, which foreshadows 'never-ending spending' and
shadow-boxing with the ghosts of electronic technology obsolescence hell.
Who worship film and video production 'shortevity' and predict doom for
production 'longevity' hell. Why should one worship 'speed greed' and
'capacity audacity', when it involves the forced expatriation of our
production asset populations just for the sake of resettlement in new and
inhospitable territory? This new religion of the technology thrill riders
(or 'thrillers') requires from its adepts, in order to get to their new
heaven, that they contribute not just tithes, but indefinite multiples of
what they have? Of course no resources will be left to generate new
productions. Its adepts will have to sacrifice that delight in order to pay
for 'the better way'! The only entertainment and enlightenment the rest of
us will then be able to enjoy, will be the agonized sorrow, the fire of
wrath and the unending grinding of teeth. Of course there will be no sound
of choirs anywhere, because the angels could not get Federal Express to
deliver the drives before they were discontinued and 'refreshed'! Chucks!

My everlasting quest for technology truths overcame me last night. I
succumbed to another temptation. It is not so difficult to identify the
failure mechanisms we could encounter when confronted with 'removable hard
disk' technology, we just need to consult the experts to test the new
doctrine and see for ourselves whether we see justification for the changed
worldview that has been preached here, or stay put. Who, I ask you, after
analysis of the poor soil, the dearth of nourishing water, the prevalence of
rocks, the heaps of ruins from past performance failure and the
tough-to-control weeds that inhabit the new land, would want to go there?

So last night, while enjoying the accomplishments of athletic giants, who
have paid their dues, I sought inspiration from C. Denis Mee and Eric D.
Daniel, in their "Magnetic Storage Handbook", published by McGraw-Hill,
1996, 2nd edition, ISBN: 0-07-041275-8, and while highlighting in yellow, we
visited some revealing places! I readily admit, it was not a high technology
'thrillride', more like a 'shiver-ride', it will be interesting to see what
the readers will feel about this new option! It did not do so much for my
enthusiasm, but more for a desire to get beyond the marketing hype and try
to evaluate on the basis of very few relevant facts, whether the new concept
could contribute new value to an archival setting. And this morning, I found
some clarifications because the book was printed way back in 1996. Computer
'technology refreshment' runs at a frequency of every 6 months (as against a
few years for video, and 100+ years for film), and a computer and its drive
is ancient in a mere three years, we need to get updated quickly!

Now any among you don't know us, we are all friends, and we do care about
the wellbeing of each other's souls. That's why we engage ewvery now and
then in some friendly banter, a sort of 'reality check', so do not take
offense please!
First, let's look at some of the terminology they use:

1. A hard disk: According to webopedia, a hard disk is: A magnetic disk on
which you can store computer data. The term 'hard' is used to distinguish it
from a soft, or 'floppy' disk. A hard disk can store anywhere from 10
megabytes to several gigabytes.  A single hard disk usually consists of
several 'platters', (also referred to as 'files'). Each platter requires two
read/write heads, one for each side. All the read/write heads are attached
to a single access arm so that they cannot move independently. Each platter
has the same number of 'tracks', and a track location that cuts across all
platters is called a 'cylinder'. For example, a typical 84 megabyte hard
disk for a PC might have two platters (four sides) and 1,053 cylinders. It
is possible to buy 'removable hard disks'.

2. A removable hard disk: A type of disk drive system in which hard disks
are enclosed in plastic or metal cartridges so that they can be removed like
floppy disks. Removable disk drives combine the best aspects of hard and
floppy disks. Their biggest drawback is that they're relatively expensive.

3. State of the art: Despite the claims made by the 'thrillers', ;-), the
largest capacity removable hard disk drive made by IBM, that uses the
'pixiedust' technology, is 120 GB (IBM Deskstar 120GXP)-See:
http://www.storage.ibm.com/hdd/desk/ds120gxp.htm. It uses a three atom thick
layer of the element ruthenium, a precious metal similar to platinum,
sandwiched between two magnetic layers, known officially as
antiferromagnetically-coupled (AFC) media. A useful tutorial about AFC can
be read here:
http://www.research.ibm.com/resources/news/20010518_whitepaper.shtml This
product has 100% greater capacity than the previous generation. It uses not
only AFC recording media, but also glass disks to ensure that the support is
smooth to support ever-shrinking bit sizes and remains flat. It includes a
number of new technologies 'for reliability', such as load/unload head
ramps, glass media, ceramic spindle motor bearings, differential read
channel, internal thermal monitor, Drive Fitness Test (DFT) and S.M.A.R.T.
Self Test technology. So at least this leaves us with the impression that it
cannot be assumed that the drive is fit, unless checked and certified to be
and remain fit. There is no data in the specifications that even
contemplates a life expectancy for the technology, which is in archival
circles an absolute "must have". It is comforting to know that there is an
interface standard :ATA-100 compatible. It has even been tested to confirm
'compatibility' with a range of other manufacturer's products. See the
summary:
http://www-3.ibm.com/storage/hdd/tech/techlib.nsf/techdocs/868811FA515DBEB87
256B0900680FOE .


4. Reliability requirements: Error rate (non recoverable): 1 in 10 -13;
Start/stops (at 40 degrees C): 40K (40,000 X) and a fascinating clincher:
Recommended power-on hours (monthly) 333 hours! Interestingly enough, IBM
have not yet established Mean Time Before Failure rates. Did they know that
Rick Prelinger would write this afternoon: "Yes, the mechanics and
electronics are a big issue. Yes, I think the MTBF figures from the
manufacturers are unrealistic." Or is Rick just preaching false doctrines?
If my calculator is right, that means that it is recommended to remain
powered for 14 days per month. So the future archive of Removable Hard Disk
drives (RHDD) may have to consist of appropriate docking stations to enable
the drives stored on the archival shelves to be turned on every other day! I
do not want to think about the alternative of hauling them to the nearest
computer every other day. There is no way in high heaven that the drives
will survive without damage to the recordings under such frequent handling.
One can imagine how this would affect the cost of video archiving on RHDD,
when in addition to this all, we would double the number of drives to avoid
loss when the RHDD crashes. To add to another clincher, it was suggested
this morning: "Also, I emphasize that two copies be made of every disk and
the second stored off site." And with his usual good sense of humor the
writer added: "I see this as a paradigm shift for archivists". I would say
it is! This afternoon I read another related thought: "Each HD can be placed
in a container just like tape, film or discs and the container is placed on
a shelf". So this would mean that when placed inside the pigeon hole bays,
the drives will have to be placed inside a container as well, and taken out
every other day! Wow, I have real difficulty visualizing that!;-).

5. Non Shakers, just Thrillers: The other interesting challenge for this
first iteration of the AFC media-based RHDD system is the specified
Environmental conditions. The reasons for this we can look at later,
obviously the micro-engineered relationship between the head and read-out
mechanisms can be damaged by shock or vibration, eh, not to speak about the
glass disks! When operating, the spec sheet states: Shock (half sine wave, 2
ms): 55G; Vibration (random (RMS)):0.67G for horizontal, and 0.56G for
vertical. And for non-operating conditions (while in the pigeon-holes):
Shock (half sine wave, 2ms): 400G (1 or 2 disks); 350 (3 disks) and
Vibration (random (RMS)): 1.04 G rms (XYZ). This implies that the allowable
force of shock during storage must not be more than 7 X its value when in
use, and the amount of tolerable vibration in storage should not be more
than twice that allowed when in use. After we have asked the high priests to
explain the meaning of this, we might put the Shock and Vibration monitors I
discussed a few days ago, to some good use, to ensure that our super duper
new storage solution does not self-destruct while being taken off the
'pigeon-holy' shelves and are transported to playback heaven! ;-).

6. 400 GB is coming: The next hallowed event is expected to occur when the
RHDD are 'expected' to store 100 billion bits (100 gigabits) of data per
square inch of disk area by 2003 (this prophesy was written by IBM on May
18, 2001)- See:
http://www.research.ibm.com/resources/news/20010518_pixie_dust.shtml In this
document IBM states that the 100-gigabit data density 'could' allow the
following capacities within 'two years': Desktop drives--400 gigabytes (GB)
or the information in 400,000 books; Notebook drives--200 GB, equivalent to
42 DVDs or more than 300 CDs; IBMs one-inch Microdrive--6GB or 13 hours of
MPEG-4 compressed digital video (about eight complete movies for handheld
devices). No information about piracy avoidance technology of course. So
relax folks, the capacity you were promised is at this stage of the hype
commonly described as 'vapor ware'. Nevertheless, it sounds pretty
attractive. But we should not forget to hold our breath, and expect some new
failure mechanism factors to rear their ugly heads once customers start
doing the testing for the manufacturers of the new 'wonder'.

7. The era of giant magnetoresistive heads: The recording technology
required the design and implementation of new magnetoresistive heads. These
heads are not developed in response to archival needs, but 'the success of
HDDs originates from an ever increasing demand for storage capacity, coupled
with a consistent reduction in price per megabyte. Improve  areal density
levels have been the dominant reason for the reduction in price per
megabyte. Oh, really? Does that mean the same trend as the decreasing
quality of floppy and recordable optical disks, made in China by previously
non-existing players? Figure 1 shows the 60% compound annual growth rate
(CDR) in areal density for IBM HDD products. If this CDR continues, areal
densities of 10 Gbits/square inch and 40Gbits/square inch are expected by
the years 2001 and 2004 respectively. IBM then lists the evolution since
1991 from the magnetoresistive (MR) head, the extended magnetoresistive
(MR)head to the follow-on giant magnetoresistive (GMR)head technologies. The
enabling technology for these newer GMR head sensors is the use of thin
films: a sensing layer, a conducting spacer, a pinned layer, and an exchange
layer. It is clear that the spacing between head and the HD storage medium
is ever shrinking, which causes one to wonder why disk crashes are
considered to be less likely, is there something we are missing here? See:
http://www.storage.ibm.com/hdd/technolo/gmr/gmr.htm . Should we assume that
these new drives will actually plug into all new computers made in the next
20 years? Really?

8. Now about past sins of HDD technology, o.k. let's call them 'HDD infancy
follies', found in the Magnetic Storage Handbook previously cited that
presented an overview of the HDD learning curve over the last 20 years:

8a. Magnetic coatings: The magnetic coatings used since the 1980s have been
Metal Particle and thin metallic alloy vacuum sputtered onto the aluminum
substrate. So the technology is actually not so different from video, audio
and data tapes.

8b. Spacing; The spacing between the platter and the reading head has
decreased from in 1979-0.73 microns, to in 1984-0.25 microns, to in 1999,
0.03 microns. That reads in my book as a higher risk of damage with 1/25th
the distance today; more risk when moving a disk in terms of shock and
vibration; more information lost for the same amount of disk surface; and
greater chance of disk crashing.

8c. Lubricant: The lubricant reduces wear with start/stop when the platter
is in contact with the slider, so there is a need for lubricant even though
the head is supposed to fly? Right, it is to enable soft landings.

8d. Track misregistration: There have surely been problems with 'track
misregistration', both 'write-to-read track misregistration', and
'write-to-write track misregistration'. (Section 2.9) The head-disk assembly
mechanical structure has a profound effect on the capability of the file to
support high track and bit densities. For instance, we have encountered
thermal track shift, resulting from misalignment of a data head from the
servo head due to thermal effects, including differences in thermal
expansion of the arms in a dedicated-servo disk file; incomplete head
settling following a track seek; apparent runout of the head-track
combination due to spindle bearing and arm vibrations at frequencies outside
the capability of the servo system; and errors in the servo position
detection circuits. Track misregistration is a random variable. In practice,
errors occur if the misregistration is greater than about 12 % of the track
width. If earlier recordings have been done to the disk, old information
data are present in the "guard band", and it contributes to data errors.
This, and failure modes of the peak-detection recording channel resulting in
peak shift, resulting from intersymbol interference, would dictate that the
RHDD to be used for archiving should not have been used before.(Section
2.13). Of course, we noted that the new system uses glass disks, rather than
aluminum, which has the unhappy characteristic of expanding more than most
other construction materials. Heat related track misregistration resulted
from the fact that the data recorded by the head at location k was recorded
when the temperature was T w,k, and read back when the temperature was T
r,k.  Typical reasons for the temperature difference would be warm-up of the
file and ambient temperature changes outside the file which affect the data
arm and the servo arm differently and likewise affect the data arm and the
data disk differently. The arms in a typical disk file are made from cast
aluminum, whereas the disks are machined from cast and rolled
aluminum-magnesium alloy, resulting in different thermal expansion
coefficients. (Section 2.16).

8e. Disk defects: Peak shift may also be caused by disk defects, random
noise, and clocking errors. You mean that a sealed cartridge can actually
still collect particulate matter, yes indeed.(Section 2.14).

8f. Signal amplitude loss: Another disk file failure mode is loss of
adequate signal amplitude which can also result from excessive intersymbol
interference, for example when a tribit pattern is recorded and the center
bit of the tribit is reduced in amplitude. These errors are referred to as
'gate errors', since the data read channel contains circuits which gate the
signal off when the read amplitude is below a specified gate
threshold.(Section 2.13)

8g. Disk slip: Repeatable runout will lead to write-over-write track
misregistration. It arises from disk slip, due to insufficient torque on the
disk clamp, or spindle imbalance. (Section 2.18)

8h. Spindle bearing problems: A significant contibution to nonrepeatable
runout originates from the spindle bearings..i.e. spindle ball bearings with
deformed inner and outer bearings races to nonrepeatable runout. (Section
2.18)

8i. Vibration: The disk files in personal computers are subjected to
external shock and vibration which can result in track misregistration even
after the servo system reduces the effect of the vibration. For example,
with a 9.5mm (3.5 in) disk file with four disks and an active servo control
system, the off track misregistration is in the range of 0.25 to 0.4 micron,
with a sinusoidal vibration of amplitude 0.5 g in the 5-1 kHz range. Yes,
they actually incorporate such drives in laptop computers as well. (See IBM
Travelstar) (Section 2.19).

8f. Ability to follow tracks: Another key performance parameter for a disk
file actuator is the ability to follow tracks. Effects that contribute to
imperfect track following are friction, stiction, and response to external
force disturbances (like windage force resulting from air flow generated by
the disks and by forces resulting from the actuator voice-coil, motor power
cable). (Section 2.23)

9. Temperature and Contamination control: Because of the power dissipation
caused by the moving air dragged along with the spinning disks, some form of
cooling is required in order to prevent excessive thermal track
misregistration. It is also essential to purge any particles away from the
head-disk interface. (Section 2.35-6).

10. Servo system performance: A servo with a roll-off of 40dB per decade
will have a lower error transfer function bandwidth than that of a system
with a roll-off of 60 dB per decade. There is a range of frequencies for
which the servo does not perform well and, in this range, it would be better
if the servo were turned off.

10a. Spindle and spindle bearings interact to produce large motions at
frequencies at which the servo can actually amplify the error. Care must be
taken in the choice of the bearings as well as in the amplitude of the error
transfer function to limit this track misregistration component.(Section
2.61)

10b. Nonrepeatable motion: Not all nonrepeatable motion is reported to the
servo loop in a dedicated-servo system. Disk flutter, dynamic actuator tilt,
and data-arm flutter all produce errors between the data heads and track
centers that are not measured and therefore cannot be corrected by a
dedicated-servo system. All mechanical track misregistration components are
acted on by the servo, with the exception of very small effects such as
surface degradtion over time (track centerline shifting due to physical
changes in the media). (Section 2.61).

11. Loss of system integrity: A loss of system integrity or impending large
positioning error, must be detected to ensure the integrity of user data,
especially true for drives (usually 2.5 in) in portable systems such as
laptops. Simply windowing the position error signal after the total position
error is less than one-half track is usually not sufficient for several
reasons: noise spikes in the position error signal would cause interruption
in the writing or reading process, degrading performance.

11a. Electrical noise disturbance forces, disk defects, and servo writing
accuracy all play important roles in determining the bandwidth and window
function trip levels used to formulate a write-ready signal.

11b. A shock to the drive (easily pictured in a laptop computer-as well when
a RHDD is moved back and forth from pigeon-hole docking station to computer)
immediately after a sample time could cause the selected head to just miss
the inhibit threshold in the following sample but far exceed it on the next.
(Section 2.23).

12. Clocking system: The clocking system has tolerances that limit
achievable recording density. The tolerance components are
steady-state-error, jitter (open-loop transient response to bit shift),
component drift in the clock circuits, frequency deviation, and adjustment
errors in the single-shot and variable-frequency oscillator. (Section 2.81).

13. Disk defects: Disk defects manifest themselves in two ways. First, there
is noise associated with the random nature of the disk surface without
defects. This noise can be separated from the rest of the noise in the
channel by measuring the noise form the preamplifier with the head flying
over the disk and then unloaded from the disk. The difference is the disk
noise. Disk defects also manifest themselves as missing bits or extra bits.
Missing bits are reductions in the amplitude of the envelope of the signal,
usually over a small number of bits (one to four), such that the amplitude
falls below the channel detector gating level. For satisfactory head-disk
assembly yields, it is necessary for the average number of defects found per
track to be less than 10% of the maximum for a reasonable expectation that
all tracks on all disk surfaces will satisfy the skip criterion. (Section
2.92).
14. Limits of error correction. The most significant measure of file
performance is the number of errors generated in a record; data on track
misregistration, peak shift, disk defects, and noise are intermediaries to
this measure. It is also important to understand how robust the design is
or, stated another way, how large a margin exists between the nominal error
rate and that which would be observed in a stressed environment (noise
reduced voltage margin, mechanical vibration, high altitude and high
temperature). The objective of assessing error rates is also to determine
specifications for head, disk, and channel parameters and for track
misregistration. (Section 2.91)

13. Stiction: Some of the factors that affect stiction are: rest time,
humidity and temperature, lubricant type and thickness, surface roughness of
both the head and disk overcoat, head slider area and load force, and
acceleration of the disk. The principal technology for reducing stiction is
texturing of the disk surface in which the disk substrate is roughened in a
controlled manner. Even after the head is flying, frictional forces are
found between the head and disk. The increase of dynamic friction is
believed to be due to the burnishing of the asperities on the disk surface
by the flying head and some accumulation of debris on the slider and by wear
of the carbon overcoat. (Section 2.132)

I hope that this little shared study of disk failure mechanisms, with a
cerful reflection on those of video and data tape and film media will help
us to have a healthy, yet open-minded attitude with regards to new
innovations that are announced in this technology and their applicability.

Finally, anyone who contemplates using this new storage technology is indeed
well advised to keep the computer it was compatible with in functional
order, as was suggested by Jim Wheeler.


Best regards,

Ed H. Zwaneveld,
Director, Technological Research and Development
National Film Board of Canada
February 21, 2002
tel: +1-514-283-9143