Here are some questionnaires, interview templates, and case studies to help measure the impact of KLS.

Questionnaires

Generic impact questionnaire

The short questionnaire provides a standard and systematic way of obtaining impact information from customers who have used a library resource or service provided by knowledge specialists.

Specific questionnaires

Questionnaire for measuring the impact of clinical librarians on direct patient care and organisational outcomes (Brettle et al. 2016)

Measuring the impact of clinical librarians on patients and NHS organisations

Questionnaire

Organisation Name and Profession

The findings of this study will be used to help your health library assess the impact of its services. The results will help in the planning of improved services that should benefit you and your colleagues in your Organisation.

The responses to the survey are anonymous, however as the survey is being carried out as part of a wider regional study, on the impact of clinical library services across the region, we would like you to indicate your Trust.

This survey is based on Best Practice Guidance on Assessing the impact of a health library service, National Library for Health, 2008 and Brettle, A., Maden, M., Payne, C. (2016). The impact of clinical librarian services on patients and health care organisations. Health Information and Libraries Journal, 33(1): 1-21 DOI: 10.1111/hir.12136

  1. Organisation Name [choose from drop down list – appropriate names to be inserted]
  2. Which best describes your profession? (Tick one only)
    Medical (including dental)
    Nursing (including midwifery, health visiting, healthcare assistants)
    Qualified scientific, therapeutic and technical staff (including healthcare scientists, Allied Health Professionals, pharmacists, paramedics and ambulance staff)
    Social care Management (including commissioning)
    Administrative support (for example, Human Resources, finance, research and development, Information Management and Technology, Governance and Risk)

This detailed questionnaire has been piloted and tested to provide impact data across the North West. It collects details of current and future contributions of clinical librarian services to a wide range of NHS related outcomes. Results of any further use of the questionnaire can be benchmarked using the data in the article.

Impact Assessment Method. See A New Impact Assessment Method to Evaluate Knowledge Resources, by Pierre Pluye et al. 2005

This is a validated scale originally developed to examine the impact of electronic knowledge resources on clinical decision-making. It has been tested and used in various populations and situations, including providing information to patients. This is a validated scale originally developed to examine the impact of electronic knowledge resources on clinical decision-making. It has been tested and used in various populations and situations, including providing information to patients.

Impact of health library on patient care (Rochester study 1992updated by Marshall et al, 2013 

This questionnaire was originally developed for one of the first studies to assess the impact of health libraries, based on the Critical Incident Technique. This short questionnaire was developed from a set of outcome based key performance indicators. It comprises 1 question: “How did the information provided by the library help?” with 5 possible responses.

Interview schedules  

The tools are designed as a best practice guide for local adaptation.  They include questionnaire schedules and a consent form for off-the-shelf use. 

Generic impact interview schedule for health knowledge and library services.

Interview schedule for measuring the impact of clinical librarians on patient care and NHS related outcomes (Brettle et al. 2016)

Evaluating the impact of clinical Librarian Services: interview schedule

This schedule was developed and tested in the study described here - Brettle, A., Maden, M., Payne, C. (2016). The impact of clinical librarian services on patients and health care organisations. Health Information and Libraries Journal, 33(1): 1-21 DOI: 10.1111/hir.12136.

Interviews were conducted by librarians from a neighbouring Trust by telephone or in person and lasted between 20 minutes and an hour. Each interview was taped and transcribed verbatim. Results were summarized using a case study template

Interview schedule

  • Date/time of interview: 
  • Interviewer: [insert name] [insert contact email/phone] 
  • Interviewee: [insert code]
  • Organisation name:
  • Professional Group: Medical (including dental), Nursing (including Midwifery, Health Visiting and Health Care Assistants), Qualified scientific, technical and therapeutic staff, Social Care, Management (including commissioning)
  • Administrative support (inclduing finance, Human Resources, research and development, Information Management and Technology and governance).

Hello, my name is XXX and I am a clinical librarian (CL) from a neighbouring organisation.

Thank you for agreeing to be interviewed today – I hope that this will take no longer than 30 minutes – is that ok? 

You have already received the information sheet about the project.  Can I confirm that you have read this and you are happy to consent to the interview, [need to create an information sheet and consent form and obtain signed form – either in person if face to face or should have received it in post for a telephone one]?
For background information and a reminder of the contents of information sheet: [adapt the information accordingly]

  • We are conducting a region wide evaluation of CL services that is looking at the impact of the information provided by CL services.  The evaluation is independent and led by Salford University in collaboration with local CLs.
  • You have been purposely selected as you have recently received information from your CL service and you represent one of the professions (or have received one of the services) on which we wish to collect information. 
  • So far – we have conducted a survey of all those who have received CL services across the region in the last 6 months.  The purposes of the interviews are to triangulate the survey data and to provide clarification and more detailed responses where appropriate.
  • Our conversation will be recorded, with your permission; otherwise I will take written notes.  These will be transcribed by an independent transcriber and analysed by members of the research team.
  • Your interview will be recorded with a number so that any statements that you make will not be attributable to you.
  • If you wish you may ask to see a copy of the transcript or notes, and a copy of the final report will be available via your library service in spring 2012.
  • You are free to withdraw your consent at any time, by contacting the researcher at the address below and your data will be removed from the sample.

Structure of the interview

The focus of the interview will be on how you have used the information on [insert critical incident] that the CL service provided.  We will be asking if the information provided has impacted on various areas in relation to patient care and organizational objectives.  At the end you will have an opportunity to add any further information or raise any additional points.

Section 1: About your request and its quality

  1. Can I clarify that you received a {insert from below} from the clinical librarian in your trust?
    Literature search.
    Literature search with critical appraisal..
    Training.
    Journal Club.
    Other.
  2. Can you briefly describe what you asked the clinical librarian to do/what role the clinical librarian to play?
    Prompt: what did you need the information for?
  3. How have you used the information?
  4. Can you indicate whether the service provided meet your needs?
    Prompt - for example was there information missing or was there anything the CL could have done to have improved/provided a better service? Such as synthesis of the literature, better presentation, less information, live links, full text of documents.
    The next few questions are about the quality of the service provided.
  5. On a scale of 1-5 (1=low and 5=high) - How relevant was the information to your request?

  6. Using the same scale 1-5 (1=low and 5=high) - How much did the service save you time? Prompt - can you estimate how much?  If the service had not been available what would you have done?

Section 2: Contribution and impact of the service provided

I am now going to ask you to what extent the information/service provided has contributed to various outcomes.  You should have already received these outcomes in my initial email.  For each one, please could you tell me if the potential information contribution is relevant?  If it is relevant has the information already contributed to the impact, May the information contribute in future or has it made no contribution to that outcome.

For all the questions below choose from:

  • not relevant
  • has contributed
  • may contribute in future
  • no contribution

Prompt - if the information made a particular contribution ask if they want to explain why or how at appropriate points.

No contribution is when the outcome is relevant to the request – but the information provided didn’t make any impact.

  • The first set of outcomes is in relation to decision-making and evidence-based practice.  Did the information contribute to?
    • diagnosis
    • choice of assessment or test
    • choice of intervention
    • advice to patient/carer
    • service development/delivery
    • revision of care pathway/guideline/protocol
    • avoidance of referral, readmission, clinical test, hospitalisation
    • reduce length of stay

Prompt - is it no contribution or not applicable. If the information made a particular contribution ask if they want to expand, for example, how has the information made a contribution, why did it contribute?

  • Did the information contribute to patient centred care and healthcare outcomes?
    • improved healthcare outcomes   
    • improved quality of life for patient or carers
    • increased patient involvement/shared decision making
    • improved patient care experience
    • improved patient access to information

Prompt - is it no contribution or not applicable. If the information made a particular contribution, ask if they want to expand.

  • The next set of questions relates to whether the information contributed to quality of care.  Did the information contribute to?
    • meeting quality standards
    • improved quality of care
    • implementing clinical guidelines/best practice
    • interventions based on best practice or current evidence
    • evaluation or audit    
    • innovative practice

Prompt - is it no contribution or not applicable. If the information made a particular contribution ask if they want to expand.

  • The next set of questions is about whether the information contributed to service development?  Did the information contribute to?
    • addressing inequalities in access to care or an unmet need in service  
    • working with other health and social care providers  
    • integrated care

Prompt - is it no contribution or or not applicable. If the information made a particular contribution ask if they want to expand.

  • Now continuing professional development (CPD).  Did the information contribute to CPD?
    • update knowledge and skills relevant to clinical practice
    • comply with requirements of mandatory or professional bodies
    • support research
    • gain qualifications
    • supervision and leadership of staff
    • delivering/supporting staff education or training

Prompt - is it no contribution or not applicable. If the information made a particular contribution ask if they want to expand.

  • The last set of outcomes is about efficiency, financial or risk management.  Has the information contributed?
    • support trust financial strategies (such as Continuous Improvement Programme (CIP))
    • support qualified institutional placement (QIPP)
    • demonstrating value for money/cost efficient service
    • risk management
    • business development (such as income generation or tenders)
    • commissioning/decommissioning services
    • improve patient/staff safety 
    •  improve accountability or transparency of services  
    • legal/ethical issues  

Prompt - is it no contribution or not applicable. If the information made a particular contribution ask if they want to expand.

  • Finally, do you want to add anything else about the contribution or impact of the library service?  Or is there anything else you’d like to add or any points you would like to emphasise? 

The end – thank you very much for your time and thoughts.

If there is anything else you think of that you would like to add, please let me know. Remember to check that the interview has recorded – if not write up your notes quickly!

This detailed interview schedule has been piloted and tested to provide impact data across the North West. It collects details of current and future contributions of clinical librarian services to a wide range of NHS related outcomes. It can be used as a stand alone tool, or to provide additional data to explain the outcomes provided from use of the questionnaire. Results of any further use of the schedule can be benchmarked using the data in the article.

Case studies  

Case studies can effectively capture and summarise the outcome of interviews. This can then be used for advocacy, marketing and promotion of library and knowledge services. We encourage you to submit your completed case studies.

Guidance on obtaining consent to use case studies.

How to write case studies.

Acknowledgements

These tools were developed by the Knowledge for Healthcare Value and Impact Task and Finish Group.  

The tools are based on  the 2009 Impact Toolkit (developed by the former NHS South Central and National Library for Health) and  the 2007 toolkit developed by the University of Aberystwyth. 

Page last reviewed: 7 December 2023
Next review due: 7 December 2024