Contents - Previous - Next
Concepts of library goodness
Michael K. Buckland
The study of "library goodness" is an underdeveloped area. There has been some speculation that there might be some general universal measure of library goodness. The idea is appealing. Imagine a Monday morning in the office as a university president, mayor or corporation chief executive officer arrives and the secretary says: "Good morning! The financial crisis is looking even worse but you'll be pleased to know that the librarian reports that the library's performance went up half a point on the library goodness scale last week." It is a nice thought but not very probable.
Single measures. of library goodness1 can be concocted but their Credibility is undermined by the number of arbitrary assumptions that have to be made to piece the parts together. Nor should this be surprising. When choosing an automobile, a variety of different factors: safety, appearance, economy, speed, comfort and so on, are considered. The problem is to relate this battery of factors to resources, intentions and personal set of values.
Although the quest for the Grail of Library Goodness has not (yet) been successful, there has been no lack of measures of performance proposed nor lack of people proposing them. The principal guide is Lancaster's Measurement and Evaluation of Library Services.2 There have been plenty of suggestions: What is lacking is coherence, a sense of the whole. It is not that there has not been progress. Lancaster's work is rather complete through about 1973, with some later work. A comparable volume written in 1963 would have been a lot thinner. Yet there is a long way to go, and it is noticeable that the numerous empirical efforts need to be counterbalanced by a greater attention to theory and to context.
There are plenty of gaps and intellectual problems in librarianship and it can be stimulating to attempt to resolve apparent contradictions. In approaching library goodness, consider three paradoxes.
The evaluation of catalogues. Books are catalogued and, therefore, retrieved by their attributes. Usually these attributes reduce to what they are "about," who their authors were, what their titles are and where and when they were published. A narrow definition of evaluation would be concerned with whether or not the catalogue (or any retrieval system) does in practice yield the items with the desired attributes.
On the other hand, it has been argued that the proper criterion for evaluation of the items retrieved should be their utility to the user, rather than their "aboutness" or other attribute used for retrieval. This sounds plausible but the utility depends on the user's state of ignorance and motivation at that particular time. So, in a sense, the catalogue would be evaluated not on its own characteristics but in terms of matters (ignorance, personal values) that are extraneous to it. This seems a little unfair, albeit desirable.
Optimal library size. In many areas of manufacturing, commerce and engineering, matters of size and scale are of central interest. The same could reasonably be expected to be true in librarianship. After all, every increment in size costs money. Yet that is not the case. The literature of librarianship is almost silent on the topic and what little there is does not get one very far and, I suspect, is little read. In brief, the literature is very limited on what might be a central concern.3
Lenin's view of public libraries in the U.S. The third paradox is quite different again. Many people connected with U.S. public libraries do not realize that a great admirer of the American public library scene was Lenin. Lenin was quite knowledgeable about libraries: His wife, Krupskaia, was a librarian.4
Library services are paternalistic in the sense that they are ordinarily provided by some for others. The appropriateness of the provision for the context in which it is provided deserves recurring consideration.
In the U.S. public libraries are viewed as a bastion of western liberal democracy and are seen as playing a significant role in free access to information, in establishing a well-informed electorate and so on. Since these are not goals generally attributed to the Soviet Union (or to Lenin), it seems paradoxical that he should have been so enthusiastic.
A discussion that can be helpful in trying to grapple with concepts of library goodness was published by Orr in the "Progress in Documentation" series of the Journal of Documentation in 1973.5
Orr points out that there is a fundamental ambiguity in discussions of library goodness because there are two quite different sorts of goodness:
Suppose that a collection of Persian prayer books was amassed and that, through assiduous purchasing and photocopying, this collection came to be the most complete collection of its kind in the world. Unquestionably this would be a good collection. If good cataloguing and knowledgeable staff were added, then we would have a good library. It would be good in the sense of quality. We can, in fact, say more than this. Quality in this sense implies capability. Such a library collection is of good quality because it is highly capable of meeting the needs of persons seeking to learn about Persian prayer books.
On the other hand, it does not necessarily follow that even the highest quality library would have beneficial effects. Let us imagine that this collection of Persian prayer books were to be in Bella Bella, B.C., or some other relatively inaccessible and sparsely populated area. What good would it do? In the absence of civilization it is difficult to imagine any beneficial effects.
Unfortunately both quality (capability) and value (beneficial effects) are difficult to measure, especially the latter. In practice we tend to fall back on surrogate measures. See Fig. 1. In particular income or resources are assumed to indicate capability: "With a book budget that low they can't do much!" or "That should be a good library, just look at the resources they have!". There is an implied causal connection. So there should be in the sense that a skilled librarian ought to be able to improve the quality of a library given improved resources. However, the improvement is not automatic any more than buying expensive ingredients guarantees a good meal if the chef cannot cook. Similarly, it is assumed if utilization is increasing, then beneficial effects are increasing. "The children's library is packed, it must be doing a good job."
Figure 1: A scheme for considering library goodness (Based on Orr.5)
These assumed connections, which are depicted by dotted lines in Fig. 1, are not unreasonable so long as it is remembered that the tightness of the connection can vary. Several things can go wrong. In particular the capability being offered may be more or less appropriate for the pattern of demand in the context where the library service is provided. We can imagine library collections more appropriate to probable demand in Bella Bella than Persian prayer books. Similarly relocating the latter in Vancouver or, better yet, Teheran would increase utilization and hence, beneficial effects.
Library services are paternalistic in the sense that they are ordinarily provided by some for others. The appropriateness of the provision for the context in which it is provided deserves recurring consideration. In this regard there is more theoretical work to be done on the demand for library services. What are the crucial attributes of demand such that the detailed profile of services provided is appropriate?
Let us imagine a situation in which the dominant form of demand is for identified documents and in which promptness of service is desirable but not critical. What profile of library service, what capability would be most appropriate? Since the emphasis is on obtaining specific, known documents, a premium should be placed on author and title catalogues. Tools of subject access play a minor, auxiliary role. Reliable document delivery is important because requests are for specific documents and, by implication, substitution of alternative titles is likely to be inappropriate. A high level of immediate availability on open-access shelves would seem desirable for convenience but, in fact, open access does not seem essential. Lengthy loan periods are tolerable if a particular item can be recalled from loan on request. Good service would be possible from closed stacks (even with shelving by size and accession number in compact storage) if documents are to be kept secure and in good order. Urgency permitting, interlibrary loan could serve this demand better than any other kind of demand. This sort of demand calls for investment in union catalogues and finding lists. The need for expert reference staff would appear to be at a minimum. ("I know what I want, please get it!")
The restraint lies outside of the library. At some point the marginal increase in the benefit to be derived from the next dollar to be spent on books is less than the benefit to the city of the next dollar to be spent on road repairs, or the benefit to the university of increasing the number of teaching assistants or whatever.
Let us now imagine what an appropriate profile of library provision might be like if we were dealing with a demand pattern characterized by specific subject inquiries and urgency. In this situation there would appear to be a premium on subject access, including both a subject arrangement of documents and subject indexes that provide additional points of access to them. The indexes might be purchased bibliographical tools or locally produced. A special emphasis on computer-based reference services and on local indexes, because of the additional power each can bring, might be expected. Indepth indexing of parts of documents would seem, in general, to be desirable, though not necessarily affordable. Experienced subject specialist reference staff or information officers can play a substantial role. To the extent that one document may be substitutable for another, low levels of immediate availability and lengthy loan periods become tolerable. Large but not necessarily exhaustive local collections within the subject area concerned are desirable to provide for browsing. Access to large holdings is also likely to be needed because the result of a subject search might become a search for one or more particular documents. In that case interlibrary loan would suffice for providing access to documents unless urgency is a major concern. The collection, even if large, ought to be on open access and arranged by subject in order to facilitate browsing even though one can also browse in subject indexes.
The two profiles of library provision that just imagined represent quite different patterns of library provision: the former resembles a traditional university library and the latter a typical special library. Other scenarios are possible. The recognizable difference between the two profiles illustrates the extent to which a library's "capability" may need to be made appropriate to the pattern of the demand to be served.6
Let us reconsider the three paradoxes in relation to. library goodness and preceding discussion.
The paradoxical situation with respect to the evaluation of catalogues ran be resolved by reference to Orr's schema in Fig.1. Retrieval evaluation in the narrow sense is a matter of quality and capability. A retrieval system that consistently and reliably retrieves just those items that have the specified attributes is, unquestionably, a good retrieval system, whether it is used or not.
The extended definition of catalogue effectiveness is concerned with the utility of what is retrieved. This approach differs from the narrow definition in two ways: Firstly, it is an evaluation of the combination of a retrieval system and its users; secondly, it corresponds to Orr's second type of goodness - what good does it do? It is concerned with value and beneficial effects. It is, therefore, different in kind from the narrow definition.7
The question of optimal library size has an explanation of a different sort. There may be circumstances in which library books ought to be relegated to less accessible storage, and there may be circumstances in which staffing ought to be increased relative to acquisitions. A change in size is a change in kind and some restructuring of the pattern of provision, such as decentralization or automation, becomes desirable. However, after all appropriate restructuring, the acquisition of one more book would seem to continue to be advantageous, even though, with diminishing returns, the advantage might become small. In other words, the marginal benefit of increased size appears to remain positive, however slight. Bigger, from this perspective, remains better.
The restraint lies outside of the library. At some point the marginal increase in the benefit to be derived from the next dollar to be spent on books is less than the benefit to the city of the next dollar to be spent on road repairs, or the benefit to the university of increasing the number of teaching assistants or whatever. We should expect the literature of librarianship to be rich in dealing with the problems of handling increases in size. However, we cannot expect it to be other than impotent in relation to optimal library size because the problem is in large measure external to librarianship and can only be resolved in relation to the context of library services.
The paradox of Lenin and American public libraries becomes less paradoxical if we review it in terms of Orr's schema and also ask about social values to define what constitute beneficial effects. It is perfectly reasonable for Lenin or anybody else to respect and admire public libraries for the capability they have to inform, educate and amuse.
The precise tuning of the capability through, for example, book selection and censorship will depend on the social values defining what beneficial effects are being sought. It is these social values that constitute the difference. Lenin was not seeking to achieve a western liberal democracy. This paradox is only a paradox so long as we fail to distinguish between the two sorts of goodness with respect to public libraries.
The concept of library goodness is ambiguous: "How good is it?" and "What good does it do?" are valid but quite different questions. Orr suggests another goodness. the goodness of library management, that would be reflected in tighter connections between the elements in his schema: more capability for any given increase in resources, more utilization for every increase in capability and so on.
Such improvement in the effectiveness of library management and in our ability to grapple with concepts of library goodness call for a greater emphasis on the theory and context of library service.
1. Hamburg. M. and others. Library Planning and Decision-Making Systems. MIT Press. Cambridge. Mass., 1974.
2. Lancaster, F. W. Measurement and Evaluation of Library Services. Information Resources Press, Washington. D.C., 1977.
3. A noteworthy exception to this neglect is Gore. D. ed. Farewell to Alexandria: Solutions to Space. Growth. and Performance Problems of Libraries. Greenwood Press, Westport. Conn., 1976.
4. Raymond, B. Krupskaia and Soviet Russian Librarianship, 1917-1939. The Scarecrow Press. Metuchen, N.J., 1979.
5. Orr. R.M. "Measuring the Goodness of Library Services: A General Framework for Considering Quantitative Measures, " Journal of Documentation. 29, 3, Sept., 1973. p. 315-332.
6. For a fuller discussion of the appropriateness of provision in relation to the sorts of inquiries served see Buckland, M.K., "Types or Search and the Allocation of Library Resources," Journal of the American Society for Information Science. 30, 3, May, 1979. p. 143-147.
7. For further discussion of uses of the term 'relevance' see Wilson, P. Two Kinds of Power: An Essay on Bibliographical Control. University of California Press, 1968, especially chapters II "Describing and Exploring" and III, "Relevance." Wilson is concerned primarily with the extended definition of relevance and utility: "textual means to an end," (p.50). In Wilson's terms the narrow definition of relevance should not be called relevance but "fitting a description, " (p. 46); "To say of something that it fits a certain description is not to employ the concept of relevance," (p.47). Michael K. Buckland is dean. School of Library and Information Studies, University of California, Berkeley. This article is based on a lecture presented at the UBC School of Librarianship under the auspices of the Universities Council of British Columbia. It was received Jan. 6 and accepted for publication on Feb. 11.
7.2 Evaluation: specific examples
A cost-analysis of cataloguing at the Universiti Sains Malaysia library for 1975
Performance measures for public libraries
The management study
I keep six honest serving-men
(They taught me all I knew)
Their names are What and Why and When
And How and Where and Who.
- Rudyard Kipling
Most librarians participate in a number of management or system studies in the course of their careers. Some conduct the studies, others are subjects, most implement the recommendations or use the findings in decision making. Even if one's involvement is limited, every professional should understand the principles and objectives of a scientifically conducted management study and know when a study is worth undertaking, who is most qualified to do it, and how best to act on its recommendations.
Who Should Make the Study
Until recently, few librarians had a sufficient knowledge of scientific management to feel comfortable in the role of systems analyst. For many years it was customary to import consultants from business and industry. Although these consultants were usually proficient in the use of analytic tools, they often did not understand library operations. They approached problems as though in an industrial setting, with a tendency to view activities solely in terms of cost. The importance of service objectives were minimized, misunderstood, or ignored.
This situation has changed somewhat in recent years as several management firms have strived to overcome these shortcomings. At the same time, more librarians have become competent in the use of analytical tools. It is now common for a large library to retain systems analysts as full-time staff members. Smaller libraries may rely on the skills of staff who also perform other responsibilities-for example, a children's librarian, cataloger, or department head who may be asked to double as an analyst whenever the need arises. Some libraries, however, still prefer to employ outside consultants.
There are advantages and disadvantages to each alternative. Regular staff members can harbor personal biases or be so caught up in internal politics that their objectivity is imperiled. Outside consultants can speak more candidly on controversial issues since they can leave the scene once the project is completed. On the other hand, staff analysts will be available to assist in the implementation of a study's recommendations after the outsiders have departed, and the ability of an organization to follow through is vital.
The ideal combination of talents is a librarian well versed in the fundamentals of systems analysis. Trained librarians are best prepared to understand what scientific-management techniques can accomplish and to recognize the limitations of these techniques in a library setting.
The Steps of a Management Study
The strategy one employs in making a management study is basically the same whatever the level of complexity. There are six steps: 1) defining the problem; 2) gathering the data; 3) analyzing the present system; 4) designing an improved system; 5) implementing the new system; and 6) evaluating the new system. The first four are discussed in this chapter, the last two in Chapter 3,
Defining the Problem
An organization usually does not contemplate a management study until it has identified a problem or deemed a change in the system necessary. It may be a member of the library staff, such as a library assistant, a department head, or the director, who brings the situation to light. In many cases the systems staff member points out the problem, often before it becomes apparent to other employees. Regardless of who first identifies the need for a study, someone in a position of authority must authorize it.
The analysis must have clearly defined boundaries: the operations (people, tasks, processes) to be studied must be clearly distinguished from those to be excluded. One should define the problem in the context of the library's overall objectives, otherwise one is liable to diagnose the problem incorrectly, For example, in recent years the profession has gradually shifted its emphasis from acquisition and organization of collections to exploitation of a library's resources. A library that is apparently able to acquire anti organize its collections efficiently may not achieve its primary objective if those materials are not also readily accessible to its readers. Incidents of such imbalances are common in libraries.
SELECTING AN AREA FOR STUDY There are several guides that an analyst can use to identify problem areas most likely to produce worthwhile results. Special attention should be given to: 1) production bottlenecks; 2) jobs that are frequently performed; 3) jobs that require frequent movement of people, forms, or equipment; and 4) jobs that require large budget expenditures.
Production bottlenecks Bottlenecks in any procedure should always receive prompt attention. A breakdown in even a seemingly minor link in a work-flow chain can be a serious matter. For example, adhering call number labels to book spines is a mechanical task; nonetheless, it is a necessary prerequisite to circulating a book. From the point of view of a user it is just as serious if a delay occurs in labeling as in cataloging, for in either case the book is delayed and the user must wait. A bottleneck can be symptomatic of several problems-staffing shortages, poor supervision, cumbersome procedures, and so on.
Jobs that are frequently performed The more often an operation is performed, the better a candidate it becomes for analysis. Even a small savings could be significant because of the high frequency. For example, suppose a public library orders 10,000 books a year; suppose further that by introducing a simple work-flow improvement one were able to reduce the average time needed for ordering by one minute per book. This seemingly insignificant adjustment would produce an annual savings of 10,000 minutes, 167 hours, or a month's labor by a full-time employee. Searching, typing book orders, classifying and cuttering books, reproducing catalog cards, filing cards, charging and discharging books, and shelving are obvious examples of high-frequency processes. In contrast, a project such as expanding a card catalog is not undertaken very often. Even though a systems study would no doubt save some time and expense for any process, a study of more frequently performed processes would produce more tangible benefits.
Jobs that require frequent movement of people, forms, equipment "Movement" is not necessarily limited to movement between two distant points. Short distances multiplied by high frequency equal long distances. Materials, forms, and equipment should be placed in the area where they will be used. Related library routines should be located so as to reflect the natural flow of the work and thereby minimize the total steps. The increase of a few steps for one person might save thousands of steps for others. This analysis of movement is inextricably tied to the study of functional library architecture. What is the optimum physical arrangement between acquisitions, cataloging, reference, the public catalog, and the major bibliographical tools? One must analyze such factors as who on the library staff uses the public catalog? How often? I low far do they have to travel to consult it? Flow long does each trip to and from it take? And so on.
Jobs that involve large budget expenditures A job that is expensive is an obvious candidate for study. High cost alone is insufficient reason for elimination or even partial curtailment. Reference service in a large library is expensive, yet who would suggest not offering it? It is reasonable, however, to seek ways to reduce a high-cost center if savings can be made without reduction in the service. For example, some libraries, upon analyzing the nature of reference questions, have concluded that many are directional and need not be answered by a professionally trained librarian. Less-costly personnel would not necessarily lower the quality of service. In today's library, computer-based systems are prime candidates for study because of their sizable operating costs.
DISTINGUISHING BETWEEN SYMPTOMS AND PROBLEMS In defining a problem an analyst should take care not to mistake a symptom for the problem. To illustrate this point, consider how people from different levels of the organization can view a problem from different perspectives. Two examples:
A Case of Uneven Work Flow
Division Head's perspective: The work flow in the department is uneven and there is a need to redesign jobs.
Section Head's perspective: The people in this section do not work at the same speed; some are very fast while others barely work up to minimum. These individuals are not carrying their share of the load.
Worker Number 1's perspective: Some of my co-workers just do not care.
Worker Number 2's perspective: All was going well until Joe started cutting out. You know, Joe doesn't give a damn about the rest of us.
The Departmental Bottleneck
Division Head's perspective: This section is not as productive as one has a right to expect; I wonder if a procedural bottleneck has developed somewhere within the unit.
Section Head's perspective: The staff do not take time to check the accuracy of their own work, so I must cover for their carelessness.
Workers' perspective: The Section Head has a compulsion to check all our work. The Section Head is a real [expletive deleted]; we are all fast, careful workers.
The manager who authorizes an investigation, on the advice of an analyst or colleagues, may decide to redefine the problem in light of additional information or terminate the investigation altogether once the problem is identified. An analyst, tot), may also recommend a change in or termination of a study as a result of learning that what was first believed to be the problem was in reality only a symptom.
Gathering the Data
One must gather data in order to document a procedure or a system. Moreover, it is this information, organized and analyzed, that will expose deficiencies and help one design, develop, implement, and evaluate an improved system. The nature of the data to be collected will vary depending upon the problem: in technical services many tasks can be described in quantifiable terms; that which is performed at a reference desk will be more difficult to measure.
What vexes an analyst is not scarcity but rather overabundance of facts, facts that must be sifted through, evaluated, anti arranged. Often the crucial task is to develop a methodology that discriminates between needed and unneeded information. There is a limit to how much can be derived from a given body, of data, and one should resist the temptation to generalize beyond its usefulness. One must also consider the costs in time anti effort to collect the data. More than one analyst has been chagrined to learn that it had cost more to gather the facts than was saved by adopting the recommended system.
One should consider the following points:
An analyst will learn from experience to judge when enough facts have been collected and the time has arrived to begin analyzing the present system.
Analyzing the Present System
Once the data are gathered an analyst should be able to model the process, problem, or system under investigation-that is, describe it precisely and understand the interrelationships among its elements. The analyst should he so familiar with the system that it could be replicated elsewhere given sufficient time and money,
The data collected Will Usually describe the work that each staff member performs or the flow of forms or materials as they proceed from one work station to another. One should become accustomed to asking questions beginning with the words why, what, where, when, who, and how.
One can use these questions to examine almost any managerial problem. A checklist (Figure 2-1) will buttress this simple approach. The analysis of data allows the manager to better understand the Current situation. Questioning assumptions and traditional practices may spark one's creative processes. Some libraries, in order to facilitate creativity, appoint task forces of qualified staff members to review the data collected during the course of a study. This provides staff with an opportunity to contribute, particularly those who may not have been directly involved in the study and who may not harbor unusual biases, but who do possess an understanding of the present system.
Figure 2-1 OPERATION CHECKLIST
Barnes, Ralph M. Motion and Time Study: Design and Measurement of Work. 6th ed. New York: Wiley, 1968. Chapters 3-6.
Johnson, Stanley, and Ogilvie, Grant. Work Analysis. London: Butterworth, 1972. Chapters 1 -3.
Mundel, Marvin E. Motion and Time Study. 5th ed. Englewood Cliffs, N.J.: Prentice-Hall, 1978. Part One (Chapters 1-6), "introduction to Motion and Time Study."
Burkhalter, Barton R. Case Studies in Systems Analysis in a University Library. Metuchen, N.J.: Scarecrow, 1968.
Burns, Robert W., Jr. "A Generalized Methodology for Library Systems Analysis." College and Research Libraries 32, 4 (July 1971): 295-303.
Chapman, Edward A. "Planning for Systems Study and Systems Development." Library Trends 21, 4 (April 1973): 479-492.
Chapman, Edward A.; St. Pierre, Paul L.; and Lubans, John, Jr. Library Systems Analysis Guidelines. New York: Wiley-Interscience, 1970.
Corey, James F., and Bellamy, Fred L. "Determining Requirements for a New System." Library Trends, 21, 4 (April 1973): 533-552.
Gough, Chester R., and Srikantaiah, Taverekere. Systems Analysis in Libraries; A Question and Answer Approach. Hamden, Conn.: Linnet, (c) 1978. Chapters 3-5.
Hayes, Robert M., and Becker, Joseph. Handbook of Data Processing for Libraries. 2d ed. Los Angeles: Melville, 1974. Chapter 6, "Methods of System Description
Heinritz, Fred J. "Analysis anti Evaluation of Current Library Procedures." Library Trends 21, 4 (April 1973): 522-532.
Minder, Thomas. "Applications of Systems Analysis in Designing a New System." Library Trends 21, 4 (April 1973): 553-564.
Contents - Previous - Next