Community legal aid clinics
Performance measures for clinics: Questions and answers
Q. How do the Clinic Information Management System (CIMS) and performance measures interrelate?
They are two separate but overlapping endeavours. CIMS will support the gathering of information to produce the performance measures. The CIMS requirements will be updated to reflect modifications when and if performance measures change.
Q. When will clinics be required to submit performance measures reports?
Clinics will be able to capture information about clinic activities and services after CIMS is implemented. Clinics will begin to submit quarterly reports beginning December 2017. At the beginning of implementation, timeframes for submitting reports will be flexible. Once clinics are familiarized with the process, the reporting timeframes will be similar to the current quarterly financial report timeframes.
Q. Why didn’t Legal Aid Ontario (LAO) work together with the Association of Community Legal Clinics of Ontario (ACLCO) to begin the first principles examination of LAO’s accountability needs as a funder and develop appropriate performance measures for clinics through a collaborative process?
LAO had to do its own thinking and identification of its needs based on its obligations under the Legal Aid Services Act, the Transfer Payment Accountability Directive, and the recommendations of the Auditor outlined in the 2011 value for money audit, prior to consultations with the clinics and the ACLCO to refine and enhance the measures.
Q. Why is LAO concentrating on quantitative measures such as average cost per case, as opposed to the quality of services provided by clinics?
Performance measures are intended to be considered as a whole. Quality of services and average cost and time allocation are equally important to LAO.
Q. The draft performance measures do not capture or reflect the complexity and systemic impact of work performed by clinics.
Initiatives now form part of the performance measures. In addition, LAO worked with the specialty clinics to develop indicators to capture the complexity, scope of impact, and results achieved. CIMS will be amended to include these indicators for both cases and initiatives.
Q. How will complexity be recorded?
LAO is working with NetDexterity to design a drop-down box in CIMS to facilitate recording of complexity factors, in line with the recommendations of the specialty clinics.
Q. Why are all performance measures at a high level?
LAO wanted to limit the number of performance measures that clinics will be required to report on, which means that measures are at a higher level. However, clinics will have the ability to obtain reports on each performance measure at a detailed level.
Q. Will LAO use the results of the performance measures to defund clinics?
No. The intent of performance measuresis to support ongoing quality improvements in clinics and client service, not to defund or reduce the funding to individual clinics or the clinic system.
Q. Why do clinics need to report on average case costs?
LAO reports on average cost per case or per assist for all its own programs, with the exception of clinics. LAO uses average case cost as a means of tracking whether LAO is achieving its strategic objectives, to budget, and to demonstrate the effectiveness of its services.
Q. Why does LAO need to implement performance measures?
Establishing performance measures is part of LAO’s obligations under the Legal Aid Services Act (LASA), the Transfer Payment Accountability Directive (TPAD), and the need to address the recommendations made by the Auditor General in the 2011 value for money audit.
Q. Clinics raised concerns that the performance measures will lead to reduced quality of the services provided to clients.
As noted above, the performance measures are intended to be considered as a whole. Quality of services and their outcome and average cost and time allocation are equally important to LAO.
Q. Will LAO set baselines and performance measures targets for the clinics?
Not at the outset. As more information is gathered and confidence in the data increases, there will be a process to establish performance targets.
Q. Will LAO use the performance measures to compare clinics to each other and to other LAO programs?
Making comparisons is a good analytical tool and will inform discussions between LAO’s regional vice presidents and clinics.
Q. Will LAO index the performance measures into a single indicator?
No. Each measure provides information about different facets of clinics’ services and provides activity data as well as measures of efficiency, effectiveness, and quality of services. Each measure will be assessed individually.
Q. Was the previous work on performance measures conducted by clinic working groups used in developing the draft performance measures?
Yes. All quantifiable measures from previous work were included.
Q. Did LAO develop a standard logic model to draft the performance measures?
Yes. The logic models are included in the documentation.
Q. Will the performance measures change as a result of the consultations?
Yes. LAO has changed the original draft performance measures as a result of feedback received to date.
Q. Some of the definitions seem unclear. Can LAO provide more detailed definitions?
Yes, definitions have been reviewed and clarified. Some refinements may be required during review of the final CIMS business requirements.
Q. How much additional work is required to input data to generate the performance measures through CIMS?
Clinics need to enter information into CIMS to manage cases. The data required to generate most of the performance measures is part of the normal work on a file or initiative.
Q. Will clinics be required to docket their time?
Yes. Time docketing is a best practice in the provision of legal services. CIMS provides an easy, quick tool for entering information.
Q. Will travel time be recorded in CIMS and factored into the cost formula?
Q. How is the hourly staff cost calculated?
LAO proposes calculation of the hourly rate based on 35 hours per week.
Q. Will voluntary and unpaid students’ time be captured in CIMS?
Yes. It will reflect a $0 cost.
Q. What components of a case will be included in the case costs?
All case activities, including disbursements form part of the cost.
Q. How will the cost of test cases performed by specialty clinics be reported?
Test cases will be docketed in CIMS. The report will differentiate between standard cases and test casesand provide information on the annual expenditure and total cost of test cases.
Q. Why is administration calculated as a residual?
LAO did not want to impose additional workload to docket administration functions. Clinics will docket for cases and initiatives, and administration will be calculated as the residual. CIMS does not provide for docketing of administrative time that is not directly related to cases or initiatives.
Q. Will the Client Satisfaction Survey be anonymous?
Q. If clinics already conduct an annual survey of clients, will they have to abandon their current survey?
No. As LAO only requires four standard questions to derive the performance measure, these can be included in the existing survey.
Q. Will the survey results be captured and included in CIMS?
No. There will be no change to the existing feedback survey process.
Q. Why is the reason for denial important?
It provides quantifiable data to identify service gaps. This information is useful to help make business cases to enhance client services.
Q. Why is LAO interested in the early resolution of files/cases?
Earlier resolution of files/cases promotes improved client services.
Q. Why is LAO interested in the resolution of cases before and after hearing?
LAO heard concerns from specialty clinics that cases to achieve law reform usually go to the hearing stage, and questioned the value of tracking whether a case goes to the hearing stage. LAO agrees, and expects these specialty clinic cases to go to the hearing stage. For most general clinics’ cases, however, an early resolution is usually of benefit to clients and reflects effective client service. LAO will track the measure and review it over time.
Q. Why was the performance measure related to initial file evaluation response time removed?
Some of the clinics consulted told LAO that clinics cannot control the time taken to find or contact a client, and cannot influence the success of this measure.
Q. Is the new Governance Scorecard replacing the annual self-assessment tool?
Yes, the new Governance Scorecard will replace the annual self-assessment tool. It is based on the previous self-assessment tool and criteria developed by Imagine Canada’s Standards Program, which is designed to strengthen public confidence in the non-profit sector. The Scorecard has the support of the Board Supports Working Group.
Q. Will LAO have access to confidential information in the CIMS database?
Q. Will reporting on clinic activities to LAO be easier than the status quo?
Yes. The CIMS implementation will ensure that reports for LAO meet clinics’ needs related to statistics and performance measures.
Q. Will clinics have the opportunity to review the reports before they are sent to LAO?
Yes. Executive directors and boards will have the opportunity to review and add qualifications to reports generated from CIMS before they are released to LAO.
Q. Will LAO rationalize information requirements from clinics?
Yes. LAO will undertake a review of all information requested and timing in conjunction with reporting of performance measures.