Name:
Ross Fuqua
State/Affiliation:
State Library of Oregon
Description of Change:
Delete data elements #502 Reference Transactions and #502a Reference Transactions Reporting Method
Current Definitions:
502 Reference Transactions (REFERENC)
Reference Transactions are information consultations in which library staff recommend, interpret, evaluate, and/or use information resources to help others to meet particular information needs. Reference transactions do not include formal instruction or exchanges that provide assistance with locations, schedules, equipment, supplies, or policy statements.
502a Reference Transactions Reporting Method (REFERRPT)
Regarding the number of Reference Transactions (data element #502) entered, is this an annual count or an annual estimate based on a typical week or weeks? Select one of the following:
- CT—Annual Count
- ES—Annual Estimate Based on Typical Week(s)
Justification:
Despite two recent attempts to fine-tune the definition for Reference Transactions, this data element continues to weaken public library data practitioners' confidence in the data and is still widely seen as among the most problematic data elements within the Public Library Survey (PLS). The inherent messiness of References Transactions data is largely blamed on how it is interpreted, tracked, and reported locally. But there are also underlying issues of local capacity, statistical validity, and the subjective and evolving nature of reference work itself.
The total number of reference transactions reported across the country has been dwindling - despite the fact (supported with anecdotal and statistical evidence) that library workers are busier than ever. In FY2010, libraries in the US and associated territories reported over 309 million reference transactions. In FY2017, that number had dropped to 240 million; by FY2022, US libraries only reported 128 million reference transactions in the PLS.
Given these trends, we believe that library workers have less time, energy, and training required to parse and track the increasingly diverse informational requests coming their way. In FY2012, 153 libraries either reported zero reference transactions or indicated that they did not track them. By FY2022, this number had grown to 254 (2.7%) libraries nationally. Also in FY2022, 52% of libraries in the US reported that they use estimates to generate the number of reference transactions locally. If over 50% of a surveyed body uses estimates for any given data point, how good is that underlying data? Whether an AE estimates or conducts an actual count of reference transactions, we believe that this data element is of very limited statistical use at the national, or likely even state, level.
Libraries may use vastly different estimation methods. Perhaps they select a few weeks a year to do an actual count of reference questions and extrapolate an annual figure based on that. Perhaps they select a single month to do an actual count and multiply it by 12. Perhaps they make guesses based on the number of phone calls, in-person interactions, emails, or chats they receive for their reference staff. Regardless, all these estimation methods result in reported statistics that almost certainly inaccurately reflect the actual reference questions received by the library. Selecting weeks or months means the estimates will vary vastly based on the time of year when those counts are taken, given the cyclic month-to-month usage of libraries. They are also subject to large changes between years if the times chosen to make the estimates are particularly slow or busy. Contacts at service points such as phone and email paint inaccurate pictures, as only a portion of those interactions meet the actual definition of reference transactions, and there is not a good way to estimate that proportion, which likely changes over time anyway.
Even the libraries that conduct actual counts of questions likely are unintentionally reporting inaccurate data. Much like estimation methods, strategies for producing actual counts vary widely. They could be counted on a paper sheet, intranet page, or separate software or a CRM, and they could be reported as they happen, after a person’s desk shift ends, or at the end of the day or even week. Library work is busy and hectic, and it can be hard for library workers to make careful counts of reference questions in such environments. Conversations with library staff who track reference transactions make clear that they routinely forget to record questions or are merely estimating anyway when they report their figures, meaning “actual counts” likely are just estimates themselves. In addition, training may vary among different types of library staff. Dedicated reference or outreach staff may have a good understanding of what constitutes a reference transaction, whereas circulation and operational staff may not have as full understanding even though they also receive questions that could be considered reference.
These deficiencies mean that this data element is statistically unsound. Reference statistics likely can’t even be validly compared among libraries within the same state, based on the different estimation and counting methods, let alone among libraries nationwide. Indeed, due to variations in methodology, time dependency, library staff, and management techniques, this data element may not even be comparable between reporting years at the same AE in many cases.
Moreover, we have concerns about the continued use of the Reference Transactions data element and how it (as a potential measurement of a library or library worker's perceived value) prolongs complicated, defensive, and biased occupational gatekeeping within our profession. Almost half of our nation's public libraries are run entirely by what some would deem as "non-professionals" (4,325 of 9,248, or 47%, of all AEs reported zero ALA/MLS Librarians on staff in the in FY2022 PLS). In FY2022, the median number of reference transactions per capita for libraries without an ALA/MLS librarian was 0.22, whereas the same median number of reference transactions per capita for libraries with an ALA/MLS librarian on staff was 0.30. We feel that while this work continues to get done in all locales, the ways in which we try to quantify it within the PLS unwittingly continues to privilege certain AEs over others. Library workers employed by libraries where an ALA/MLS is required of librarians might be more pre-disposed to demonstrate and quantify the perceived value of that training, thereby painting an inaccurate picture of reference nationwide.
We understand that reference work is a core library function, and library workers understandably want to quantify that work in some way. For that reason, we understand that libraries and even individual states may want to attempt to quantify reference transactions in some way, however flawed. But library reference work is inherently messy and defies quantification. What constitutes a “reference transaction” is a far more subjective question than what constitutes a library program, collection material, or Wi-Fi session. Ultimately, by presenting this data element as a part of the national PLS dataset, we are saying as state data coordinators that we believe it is statistically valid information that can be compared among libraries nationwide. It almost assuredly cannot. For these reasons, we propose to eliminate this as a nationwide data element and leave it up to individual libraries or states on whether and how to track library reference work.
Potential methodological issues:
None
Name:
Ross Fuqua
State/Affiliation:
State Library of Oregon
Description of Change:
Delete data elements #502 Reference Transactions and #502a Reference Transactions Reporting Method
Current Definitions:
502 Reference Transactions (REFERENC)
Reference Transactions are information consultations in which library staff recommend, interpret, evaluate, and/or use information resources to help others to meet particular information needs. Reference transactions do not include formal instruction or exchanges that provide assistance with locations, schedules, equipment, supplies, or policy statements.
502a Reference Transactions Reporting Method (REFERRPT)
Regarding the number of Reference Transactions (data element #502) entered, is this an annual count or an annual estimate based on a typical week or weeks? Select one of the following:
Justification:
Despite two recent attempts to fine-tune the definition for Reference Transactions, this data element continues to weaken public library data practitioners' confidence in the data and is still widely seen as among the most problematic data elements within the Public Library Survey (PLS). The inherent messiness of References Transactions data is largely blamed on how it is interpreted, tracked, and reported locally. But there are also underlying issues of local capacity, statistical validity, and the subjective and evolving nature of reference work itself.
The total number of reference transactions reported across the country has been dwindling - despite the fact (supported with anecdotal and statistical evidence) that library workers are busier than ever. In FY2010, libraries in the US and associated territories reported over 309 million reference transactions. In FY2017, that number had dropped to 240 million; by FY2022, US libraries only reported 128 million reference transactions in the PLS.
Given these trends, we believe that library workers have less time, energy, and training required to parse and track the increasingly diverse informational requests coming their way. In FY2012, 153 libraries either reported zero reference transactions or indicated that they did not track them. By FY2022, this number had grown to 254 (2.7%) libraries nationally. Also in FY2022, 52% of libraries in the US reported that they use estimates to generate the number of reference transactions locally. If over 50% of a surveyed body uses estimates for any given data point, how good is that underlying data? Whether an AE estimates or conducts an actual count of reference transactions, we believe that this data element is of very limited statistical use at the national, or likely even state, level.
Libraries may use vastly different estimation methods. Perhaps they select a few weeks a year to do an actual count of reference questions and extrapolate an annual figure based on that. Perhaps they select a single month to do an actual count and multiply it by 12. Perhaps they make guesses based on the number of phone calls, in-person interactions, emails, or chats they receive for their reference staff. Regardless, all these estimation methods result in reported statistics that almost certainly inaccurately reflect the actual reference questions received by the library. Selecting weeks or months means the estimates will vary vastly based on the time of year when those counts are taken, given the cyclic month-to-month usage of libraries. They are also subject to large changes between years if the times chosen to make the estimates are particularly slow or busy. Contacts at service points such as phone and email paint inaccurate pictures, as only a portion of those interactions meet the actual definition of reference transactions, and there is not a good way to estimate that proportion, which likely changes over time anyway.
Even the libraries that conduct actual counts of questions likely are unintentionally reporting inaccurate data. Much like estimation methods, strategies for producing actual counts vary widely. They could be counted on a paper sheet, intranet page, or separate software or a CRM, and they could be reported as they happen, after a person’s desk shift ends, or at the end of the day or even week. Library work is busy and hectic, and it can be hard for library workers to make careful counts of reference questions in such environments. Conversations with library staff who track reference transactions make clear that they routinely forget to record questions or are merely estimating anyway when they report their figures, meaning “actual counts” likely are just estimates themselves. In addition, training may vary among different types of library staff. Dedicated reference or outreach staff may have a good understanding of what constitutes a reference transaction, whereas circulation and operational staff may not have as full understanding even though they also receive questions that could be considered reference.
These deficiencies mean that this data element is statistically unsound. Reference statistics likely can’t even be validly compared among libraries within the same state, based on the different estimation and counting methods, let alone among libraries nationwide. Indeed, due to variations in methodology, time dependency, library staff, and management techniques, this data element may not even be comparable between reporting years at the same AE in many cases.
Moreover, we have concerns about the continued use of the Reference Transactions data element and how it (as a potential measurement of a library or library worker's perceived value) prolongs complicated, defensive, and biased occupational gatekeeping within our profession. Almost half of our nation's public libraries are run entirely by what some would deem as "non-professionals" (4,325 of 9,248, or 47%, of all AEs reported zero ALA/MLS Librarians on staff in the in FY2022 PLS). In FY2022, the median number of reference transactions per capita for libraries without an ALA/MLS librarian was 0.22, whereas the same median number of reference transactions per capita for libraries with an ALA/MLS librarian on staff was 0.30. We feel that while this work continues to get done in all locales, the ways in which we try to quantify it within the PLS unwittingly continues to privilege certain AEs over others. Library workers employed by libraries where an ALA/MLS is required of librarians might be more pre-disposed to demonstrate and quantify the perceived value of that training, thereby painting an inaccurate picture of reference nationwide.
We understand that reference work is a core library function, and library workers understandably want to quantify that work in some way. For that reason, we understand that libraries and even individual states may want to attempt to quantify reference transactions in some way, however flawed. But library reference work is inherently messy and defies quantification. What constitutes a “reference transaction” is a far more subjective question than what constitutes a library program, collection material, or Wi-Fi session. Ultimately, by presenting this data element as a part of the national PLS dataset, we are saying as state data coordinators that we believe it is statistically valid information that can be compared among libraries nationwide. It almost assuredly cannot. For these reasons, we propose to eliminate this as a nationwide data element and leave it up to individual libraries or states on whether and how to track library reference work.
Potential methodological issues:
None