Contents - Previous - Next


7.6 Data density versus data security: formats suitable for archival purposes

Dietrich Schüller

Physically the recording of audiovisual material of all kinds (sound records, photographic stills and moving images, videograms) means to store an amount of data which is related to sound pressure-waves, or to emitted or reflected light. The problem common to all kinds of recordings lies in the fact that an enormous amount of data has to be written onto or read from the storage medium - in most cases in real time. The storage medium itself has to keep the data undistorted over a long period of time, ideally to eternity. It should be easy to handle and its size should be practicable.

We should try to discuss our problem by dealing mainly with sound records, taking a glance, however, at electronic processing and storage of images, but leaving aside film and photography for the moment. If we look at the history of av-records we discover two parallel developments: an ever increasing quality, which means an ever increasing amount of data, accompanied by an ever decreasing size of carrier, and an increasing data density. As an example take the bandwidth of sound recordings:

1900: acousto-mechanical 5 kHz
1925: electromechanical 10 kHz
1960: magnetic 20 kHz

Parallel to this, space consumption has been reduced drastically if comparison is made between the volume required to store 1 hour of recording in various formats (boxes included):

2 minute cylinders 11 000 cm3
analog tape
(38 cm/s on 26.5 cm spools) 12 665 cm3
R-DAT (2 hours) 76 cm3

More recently during the last 15-20 years, together with increasing data density, there is a tendency towards decreasing recording quality in formats aimed at the average consumer, and the compact cassette and the home video formats represent good examples of this tendency. On the digital side we see strong moves towards data reduction techniques which often impair technical quality as well.

Leaving aside these trends towards lower quality standards for the average consumer any development towards a certain point unavoidably leads to insecurity of data storage, on the carrier itself and on its readability.

There are several reasons for this:

1. Miniaturization of wave-lengths and recording tracks call for extreme precision of recording and playback equipment and for almost perfect and uniform physical and chemical condition of the recording media. Extreme cleanliness and special environmental conditions are absolutely essential.
2. Thinner carriers. especially tapes, exhibit lower mechanical stability with all its consequences.
3. Magnetic carriers with high packing rates, eg. the high percentage of magnetic oxide, sometimes results in chemical instabilities of the oxidebinder compound, especially under unfavourable climatic conditions. Beyond a certain point data density is inversely proportional to data security. Data density is dependent on the occupied a (ignoring the fact that all kinds of data storing systems are, strictly speaking, three dimensional): the smaller the area, with constant amount of data - the greater the density and the greater the problems arising therefrom. Furthermore, it appears that function is a non-linear one.

In using and handling audiovisual carriers, however, access areas and volumes which cannot be used to store information are necessary. For example the area occupied by the label on a record, the hubs, spools or cassettes for magnetic tape, the covers and boxes to house the carriers properly. For obvious reasons, these areas and volumes cannot be reduced beyond a certain point. Record covers should be big enough accept notes which can be read comfortably. Cassettes occupy a certain volume especially if they have to protect a sensitive carrier. Another factor adds to this complication: a minimum shelf thickness and adequate spaces are required to place the carrier onto the shelf or to remove it. However small a cassette may be designed, there will always be the need for at least an extra 3 cm (one for the shelf and two for the fingers) for handling. Most obviously open shelving in an archive with at least a 60 cm aisle width is sometimes in absurd contradiction to the efforts of miniaturizing data carriers, but moving shelves or other devices for the optimal use of storage areas is extremely costly and prolongs the access time.

Data density and storage space are therefore in non-linear relation to each other.

Additionally, some aspects of practicality and comfort have to be observed. Small carriers are more likely to get lost or stolen than larger ones. Small carriers also suffer from limited space for written information; simply compare the notes on an LP with those of the same contents on a compact cassette in terms of elegance and comfort.

Summary

The unreflecting increase of data density as a goal in itself is senseless. Up to a certain point smaller carriers are practicable and desirable especially by those who are on the production side (such as the ENG people). Beyond a certain point, however miniaturization becomes questionable. The superficial advantage in terms of storage requirements and the building and maintenance costs involved have to be offset against the costs arising from problems of possible data insecurity. Economists are challenged to calculate the costs of long term preservation including the costs of subsequent transfer of whole archives to new formats. Such a calculation may prove that radical miniaturization may in the end not be the most economic way to store audiovisual material over long periods. This calculation would also have to consider forthcoming obsolescence of the highly sophisticated equipment necessary for future formats. While it is still feasible to construct replay machines for Edison cylinders with reasonable effort, it is questionable whether future equipment may be rebuilt once mass production has ceased.

Archivists therefore would prefer formats which optimize data security rather than data density. Contrary to general opinion a far higher percentage of all data carriers produced worldwide is retained for at least medium-term storage. This trend is more and more increased by the growing interest in the history of cultural life and science all spheres. Therefore, in developing new formats, more attention should be paid to the aspects of long-term preservation.

The following comments to the paper from many experts in the field is included for information.

William D. Storm: I would like to make a comment as much as anything else. What I am particularly concerned about when we come to data storage and the idea of giving more and less space, is to argue whether material could be packaged in a different way, with a larger area for labelling, but very small actual storage area People are losing quality because they want to have something that is small and this scares me more than anything else. You get compressed data, and compressed data to my way of thinking is insufficient and very poor data.

Dietrich Schüller. I quite agree upon the aspect you raise on quality. We have, however, to admit that other factors are obviously pushing development forward, one being the notorious lack of space of many archives, normally located in the centres of cities But I think that further miniaturisation of recording carriers will only seemingly solve this problem while it introduces others. Therefore we have to calculate storage space into the future and against any financial inputs we have to make for the future care of things. Although I have not calculated it yet, my guess is that storage space is easier to finance than even one extra generation of rerecording. If you can save one generation of rerecording in the next 100, 200 or 400 years then it is cheaper to pay for greater storage area than to leave the troubles to your successors.

Jean-Marc Fontaine: I would like to contribute with a reflection, Dietrich, as the problems of dimension is a fundamental question now brought about by numerisation of sound. I think we will have to stay calm in the years to come, as we will have a large numbers of propositions as to the supports for storage. Today we touched on R-DAT, and I would nevertheless like to reflect on Mr. Wheeler's expression concerning storage; this is a product that does not yet exist and we have to assess it, knowing that it is a cassette, a reduced cassette, and analogy has taught us at least to be cautious. Next year we will be talking about CD-ROM, rerecordable disks, and gradually we will get the propositions and have to reflect a lot on these, once again calmly.

Dietrich Schüller: My paper was intended to be a first voice from the archivists' side against the standards which are imposed on us by industry which does not construct the products for archival purposes. This forum and especially the Unesco Consultation which follows, is very important, because it is a chance for the first time to combine the voices of the archivists who have to pay in the long run for the future care of those products which are on the market now. It will lead to some interesting results.

Jim Wheeler. Thank you. Dietrich I will hit you from both sides. Actually I compliment you for taking a stand like this. The Engineering Societies do not hear this kind of a voice, you are right. I think you should be speaking to the Society for Motion Pictures, the Television Engineers Society and the European Broadcasting Union, rather than this group, but I think that some of your assumptions are wrong. One of them is the high packing density: what is happening is, the magnetic heads are being made (I am also a head design engineer, and design tape, heads, the rotary scanner and all this kind of work) anyway in the heads we are making we use very different materials in recent years than before, so we are able to make smaller heads which are much more precise. The metal particle tape allows a much higher packing density, we reckon with a much smaller particle on the tape. Another point you have is on the thin tape. I have been concerned about thin tape, at AMPEX I raised the flag, and was made responsible to make sure that the machines work with thin tape. Provided you use a cracked tension, I do not see that the thickness of the tape is so critical a factor, although the point may be debatable. On getting smaller you are talking about shelf space, but you are not being innovative in thinking of the future when you will have an automated archival system at whatever point you want to enter the system, a cherry-picker, or whatever you call it in Europe. This will pick out the cassettes and deliver them to the person who wants them. With computer control, where no human hands are involved in the system, you have to be thinking differently, not of shelving as you have in the past. On miniaturization however you have a good point to make.

Dietrich Schüller: Everybody admits that with increasing data density we run into trouble, but we should overcome this trouble. I am not so sceptical as I may have sounded, but we have to calculate what the present and the future costs are. I know of course that we could have automated archives, where materials are retrieved automatically. But all these systems (R-DAT and so on) run into the problem of obsolescence, how do we get these machines in 100 or 400 years time? Today we have seen that we are able to construct cylinder replay machines for an economic input which are far better than machines from the earlier era. It will be virtually impossible economically to construct any of those machines needed to replay all these standards in 500 years time. That is the problem which we have not been discussing, but it is impossible in practical term to re-make all these alloys or build them up again with discrete elements. There is a second great problem which we have not touched - we should be thinking about a format which is easy to handle. Why does R-DAT run for two hours, not for one hour? Why is it a .15 in tape, why is it not an 8 mm tape, if this is available on the market? These are all things which I do not understand: It would cost a little bit more shelf space but it would be four times more practical - that is the message of the paper.

Jim Wheeler: What pushes the market here is what sells in TV stations, not what sells to archives. That is the driving force, it is called money. So you should be speaking to the Engineering Societies to make the point that you do need a common format, to settle on a format. I am not sure how you handle that, but some organisation from this meeting should be approaching the Engineering Societies in the world.

William D. Storm: A further comment. There is a very important technical concern here. Archivists are supposed to have to live with what the manufacturers do, but there may be a basic flaw in this thinking. If there is a body which shows what they are doing, not what they advertise they are doing I think you can greatly influence the market as well. Tape, for example: there are many different types of tape that vary tremendously from batch to batch as much as 6 decibels. Thin tape is definitely nowhere near as good as the thicker tapes, and this is very measurable, very qualifiable, however the general market is not aware of that. When the archivists start to become more like scientists and show what is really happening, not the market hype, we can indeed influence what we are given and what we have to work with.


Contents - Previous - Next