I was inspired by Munday’s writing on problems and mysteries, and by the project by a PGCert alumni that investigated what students perceived by the phrase “intellectually stimulating” to look into what mysteries might lie within the library NSS results and how they could be investigated.
The NSS Library Question
There is one question on libraries in the NSS, Question number 19.
Previous to 2017 the library questions was:
“The library resources and services are good enough for my needs”. (NSS Question changes for 2017)
In 2009 Tracey Stanley gathered feedback from librarians on the library question and gained this feedback:
- “It does not focus on the electronic aspects of the library service, which play an increasing part in overall service provision.
- The phrase ‘good enough for my needs’ appears to encourage a negative response.
- The question does not allow respondents to focus their response on satisfaction with either stock or services.
- The question is linked with others also included under ‘Learning resources’ – most significantly, a question about access to specialised IT resources – which may skew the overall result for the ‘Learning resources’ section as specialised facilities may not, by their nature, be widely available to all.
- It does not focus on the level of support for learning provided by the library service.”
I disagree with this in one respect: I think the phrase “Good enough” implies mediocrity and sets a very low bar, especially compared to the loftier tone of some the other questions, which ask students to consider whether their course has been intellectually stimulating, has challenged them to achieve their best work and explore concepts in depth. Finding that most students think the library service is “good enough” doesn’t seem to be something we should particularly celebrate, as many libraries have done. The University of Oxford even made the rather spurious claim “Oxford libraries voted best in the UK for second year running.” when they achieved the highest score nationally on the library question in 2013 and 2014.
The question was improved for the 2017 survey to the following:
“The library resources (e.g. books, online services and learning spaces) have supported my learning well.” (National Student Survey)
Although this is still somewhat problematic, as the question is still linked to other questions on specialist facilities and does not allow respondents to distinguish between library resources and services, the data yielded from this might be more reliable in telling us if the students feel the library is really doing a good job in supporting them in their studies.
This appears to be a very good result, but it is too easy to be satisfied, there are still many questions to be asked of this data. I have listed a few below, and how they could be investigated.
What do students understand by the question?
What do students understand by the phrase “supporting my learning”? How do they think the library should be doing this? What are their expectations of the library? Is it still possible that the students have low expectations, perhaps because they are not aware of all the services the library offers? It would be interesting to conduct focus groups with students to ascertain what they understand by Question 19, and what their expectations of the library service are.
What are students’ perceptions of Library Services?
Recently Library Services conducted a Digital User Experience project where students were asked to map their research processes. It was found that some students were unaware of what the role of the library staff is. Some were confused as to whether we are there simply to manage the space and collections, and monitor behaviour, or whether we were there to help them. This reticence to ask questions certainly indicates there are low expectations of the ability of library staff and will inevitably mean there is a lack of awareness of library resources and services.
This 1500 CSM students attended a library information skills session in 2016-17 (Information skills can include how to use the online resources, database searching, evaluating the reliability of resources, referencing skills etc.) and only 105 CSM students attended a one-to-one library tutorial. Many courses do not take up the offer of information skills training from their subject librarian, and the take-up of library tutorials is relatively low, so students may never be aware of the online resources provision and never have the opportunity to acquire more advanced research skills.
It seems possible that the highly positive NSS results could be actually be the outcome of lack of awareness and low expectations of the library service. As well as focus groups, we can look at other measures available. The results from the Library Survey that took place in the Autumn of 2016 give a fuller picture of how the students and staff view the various aspects of the library service, and as shown in this infographic, it is a more complicated picture than the NSS results suggest.
Could the good results be masking a problem?
Although only 9% of students across UAL did not agree that the library supported their learning, it would be interesting to find out more about that 9%. What if disproportionate numbers of students from lower socio-economic backgrounds, BAME students, international students, or students with disabilities felt that the library was not supporting them? I can not discern this from the dashboard data as it only breaks down each question group, not individual questions, by student profile. It would be interesting to see responses to question 19 across UAL by student profile to see if there are any groups that do not feel the library supports them. If there was, this would then require further investigation to see why. Mainly, it is important to remember with statistics that a positive result can mean you are serving the majority well, while some groups may not be being served well at all.
As we wouldn’t think that some bad feedback meant that everything we are doing is wrong, it is important not to see good feedback as an endorsement of everything we do. We must look beyond the data to ensure that there are not bad reasons for good feedback.