Performance measures: examples of charts and reports

Once the Clinic Information Management System (CIMS) has been implemented, clinics will be required to submit quarterly reports on their performance measures. Reporting of specified measures is mandatory for all clinics.

Purpose of performance measures

These clinic performance measures have been developed to meet LAOs obligation under the Legal Aid Services Act and the Transfer Payment Accountability Directive, and address recommendations made by the Auditor General in his 2011 value for money audit. These performance measures are to be used by LAO and the clinics as a management tool, to provide the data for organizations to gauge whether they are achieving their goals and objectives in an effective and efficient manner. The reports should inform decisions and promote continuous improvement and fulfil obligations for accountability and transparency to the public. While the performance measures were designed to meet LAOs legislative requirements, they will evolve over time and will provide a foundation for discussions between LAO RVPs and the clinics. You can get more detail on the development and intended use of the measures on the Legal Aid Ontario website.

The first full-year report will be available for the 2018/19 fiscal year. Development of reports are in progress to be delivered post CIMS go-live.

1. Performance measures generated by CIMS

LAO did not want to impose additional workload on clinics to docket administration functions. Clinics will be docketing for cases and initiatives, and administration will be calculated as the residual. Administration includes all activities not considered to be direct client services.

1.1 Measure #1: Cases and initiatives: services provided, average cost per case, and cost of initiatives

1.1.1 Sample reports Cases/Initiatives

Standard case summary data for reporting period Total
Open cases at beginning of reporting period 425
New cases opened during reporting period 1,575
Total cases open during the period 2,000
Total cases remaining open at the end of the period (=active) 480
Total cases closed during period 1,520
Total cost of cases closed during period $577,600
Closed cases with flagged complexity* indicator 812
Referrals during the period 900
Date of oldest open case Jan 15, 2015
Test case summary data for reporting period Total
Open cases at the beginning of reporting period 3
New cases opened during the reporting period 1
Total cases open during the period 4
Total cases remaining open at the end of the period (=active) 2
Total cases closed during period 2
Total cost of cases closed during period $320,000
Average cost of cases closed during period $160,000
Cost summary data in reporting period Total
Total cost of all standard case work in period $180,000
Total cost of all test case work in period $200,000
Total cost of all initiatives in period $120,000

The following table is based on the primary area of law selected in CIMS. The performance measures only report on these primary areas of law.

Standard case area of law Standard cases opened during period Standard cases closed during period
Housing 300 320
Social assistance 206 230
Workers compensation 550 600
Immigration/refugee/ citizenship 94 34
Employment 265 201
Other types of law 160 135
Total 1,575 1,520
Standard and test cases closing during period - Results achieved1 Number of standard cases Number of test cases Comments*
Individual client/family
-liberty 13    
-monetary award value 28    
Total $ amount if monetary award $46,000    
-income maintenance 34    
-secure housing 5    
-discrimination addressed 8    
-enhanced client independence 2    
Other 0    
Number that resulted in change in legislation   2  
Number that resulted in change in regulations   1  
Number that resulted in change in Common Law   3  
Total cases 42 6  

* Provide comments in performance measures submission report template.

1.1.2 Efficiency measure

Standard cases closed during period by primary area of law Number
Housing cases 320
Social assistance cases 230
Workers compensation cases 600
Immigration/refugee/ citizenship cases 34
Employment cases 201
Other types of law cases 135
Total cases (closed) 1,520
Average time to close case (months) 6
Average cost per closed case $380
Standard case primary area of law Average cost per standard case
Housing cases $425
Social assistance cases $260
Workers compensation cases $180
Immigration/refugee/ citizenship cases $640
Employment cases $250
Other types of law cases $300
All closed cases $380
Cost by initiative types during reporting period Total cost in period
Public legal education/outreach $32,000
Training $14,000
Community development -
Policy advocacy $2,300
Law reform $5,000
Systemic advocacy $8,000
Partners/network/community groups -
LAO/clinic committee & consultations -
Inter-clinic groups $4,500
Memberships -
Media/communications $12,000
Governance $12,000
Professional development -
Other $2,050
Total initiatives $91,850

1.1.3 Effectiveness measure

Measure based on standard cases closed in two previous fiscal years and current reporting period 2011 2012 Current reporting period
Average cost per standard case* $300 $320 $380

			Average cost per standard case. Chart shows 
			$300 in 2011, 
			$320 in 2012, 
			and $380 in 2013.

1.2 Measure #2: Resource allocation

The measures below will be achieved through docketing.

1.2.1 Sample data

Hours docketed for open and closed cases (during period) Number
Intake/referral/other service (2 hours or less of time docketed) 462
Standard case work (over 2 hours of time docketed) 5,950
Test case work (over 2 hours of time docketed) 1,130
Initiatives 5,300
Direct client service (sub-total) 12,842
# of FTEs 9
Total hours = 9 FTEx1561hrs 14,049
Administration and other (not docketed, but by subtraction), includes administration plus other activities such as continued educational learning, admin tasks, filing, etc.) 1,207
# of referrals 1,450

1.2.2 Efficiency measure

Resource allocation 2011 staff time (%) 2012 staff time (%) 2013 staff time (%)
Intake/referral/other service (two hours or less of time docketed) 4 4 3
Standard case work (over two hours of time docketed) 39 30 42
Test case Work (over two hours of time docketed) 15 12 8
Initiatives 31 41 38
Administration and other 11 13 9

				Pie chart for Resource allocation in 2013. Chart shows 42% standard case work, 38% initiatives, 9% administration and other, 8% test Case work, and 3% for intake/referral/other service.

Chart 2. Resource allocation - 2013

1.2.3 Effectiveness measure


				Bar chart for resource allocation. Chart shows Intake/referral/other service as 4% in 2011, 4% in 2012, and 3% in 2013. 
				Standard case work was 39% in 2011, 30% in 2012, and 42% in 2013. 
				Test case work was 15% in 2011, 12% in 2012, and 8% in 2013. 
				Initiatives were 31% in 2011, 41% in 2012, and 38% in 2013. 
				Administration and other was 11% in 2011, 13% in 2012, and 9% in 2013.

Chart 3. Resource allocation

1.3 Measure #3: Service outcomes and client feedback

1.3.1 Efficiency measure

This is based on closed standard cases (over 2 hours of time docketed). The information will be captured when the case is closed.

Note: Withdrawn/discontinued = matter is withdrawn or legal action is discontinued by the clinic or client.

Service outcomes of closed standard cases in 12-month period as perceived by clinic staff # of cases
Successful (meets clients objectives) 625
Partially successful (somewhat meets clients objectives) 221
Unsuccessful 225
Withdrawn/discontinued 99
Unknown 36
Total 1,206
Service outcomes of initiatives completed in 12-month period, as perceived by clinic staff # of initiatives
Successful 3
Partially successful 1
Unsuccessful 2
Discontinued 1
Total 7

1.3.2 Efficiency measure

Service outcomes of standard cases as perceived by clinic staff # of cases Cost per closed case
Successful (meets clients objectives) 625 $1,750
Partially successful (partially meets clients objectives) 221
Unsuccessful 225 $1,283
Withdrawn/discontinued 99
Unknown 36

				Average standard case cost by service byoutcome for 2013. Chart shows successful / partially successful at $1,750, and unsuccessful/withdrawn or unknown at $1,283.

Chart 4. Average standard case cost by service - Outcome 2013

Service outcomes of standard cases as perceived by clinic staff Average cost per case, 2011 Average cost per case, 2012 Average cost per case, 2013
Successful or partially successful $1,535 $1,620 $1,750
Unsuccessful, withdrawn, discontinued, or unknown $1,380 $1,300 $1,283

				Cost per standard case by outcome for 2011 to 2013. Chart shows the average cost per case for successful / partially successful at $1,535 in 2011, $1,620 in 2012, and $1,750 in 2013. 
				For unsuccessful/withdrawn or unknown the chart shows $1,380 in 2011, $1,300 in 2012, and $1,283 in 2013.

Chart 5. Cost per standard case by outcome (2011-2013)

1.4 Measure #4: Served versus denied and referred

1.4.1 Sample data

Intakes served and denied during reporting period # %
New cases opened (proxy for clients served) 2021 80%
Not financially eligible 186 7%
Not part of the clinics area(s) of law 52 2%
Services not offered by clinic 92 4%
Not within catchment area 138 5%
Conflict of interest found 14 0.6%
Other 26 1.1%
Total denied or not eligible 508 20%
Grand total 2,529 100%

1.4.2 Analytical information


				Percentage of client services vs. denied service or not eligible. Pie chart shows 80% for clients served and 20% for clients denied service or not eligible.

Chart 6. Percentage of clients services vs. denied service or not eligible

1.4.3 Effectiveness - to be calculated from previous table

Year of service # of standard cases opened # of intakes denied or referred Total Percent cases opened to total
2011 2,062 688 2,750 75%
2012 1,900 660 2,560 74%
2013 2,014 520 2,529 80%

				Bar chart for standard cases opened as a percent of total intake in 2011-2013. Chart shows 75% in 2011, 74% in 2012, and 80% in 2013.

Chart 7. Standard cases opened as a percent of total intake, 2011-2013

1.5 Measure #5: Stage when case file outcomes are achieved with clinic involvement

1.5.1 Sample data

Resolution stange # of closed standard cases %
Before hearing 3,500 64%
After hearing 2,000 36%
Cases withdrawn or lost contact 165  

1.5.2 Efficiency chart to be calculated from data in 2.5.1 and 2.5.3

Resolution stange # of closed standard cases Cost per standard case
Before hearing 3,500 $1,725
After hearing 2,000 $3,500

				Average standard case cost by resolution stage for 2013. Bar chart shows $1,725 for before hearing and $3,500 for after hearing.

Chart 8. Average standard case cost by resolution stage (2013)

1.5.3 Effectivness

Year of standard case closing Average cost If standard case closed before hearing Average cost if standard case closed after hearing
2011 $1,700 $3,200
2012 $1,860 $3,700
2013 $1,725 $3,500

				Average standard case cost by resolution stage for 2011-2013. Bar chart shows before hearing with $1,700 in 2011, $1,869 in 2012, and $1,725 in 2013. After hearing shows $3,200 in 2011, $3,700 in 2012, and $3,500 in 2013.

Chart 9. Average standard case cost by resolution stage, 2011-2013

1.6 Definitions

Admin time*

Admin time for a time period is the residual time calculated by taking the time docketed for referral and brief service, standard case work, test case work, and initiatives from the total hours of effective staff time calculated from the FTE available for the time period.

Annual percentage resource allocation

For section 2.2.2, the annual percentage resource allocation, the percentage of staff time per activity type (referral and brief service, standard case, test case, initiatives, and admin time), are calculated as the percentage of the total hours docketed during the year against each of the four categories compared to the total effective annual hours available for all clinic staff.

Areas of law

Areas of law in which a clinic may provide service. See the complete Area of Law list in CIMS.

Case file

The record of the clinics response to a clients request for assistance. This includes the services offered, work completed and relevant details and notes about a clients matter. A Case File is not created for situations where the clinic has only offered a referral elsewhere for service or has declined to provide services.

Case file closing

Date when all matters laid out in the retainer agreement (written or verbal) have been completed.

Case file outcome

Result of clients matter. For example: hearing order, settlement, eviction, non-eviction,-financial recovery, immigration status settled.

Case file work

Case file work includes legal research, drafting submissions, preparing for and appearing at hearings/court, travel, client meetings, case conferences, negotiations, letters, document-drafting, scheduling appointments, etc.

Client

Persons served by any clinic staff. Person to whom a clinic has agreed to or is obligated to provide a service or anyone to whom a lawyer owes a duty of confidentiality whether or not a solicitor/client relationship exists.

Client objective

Clients expected/desired remedy to the matter

Conflict of interest

Whenever confidential information is received from a prospective client, identifying information is checked against the clinic client database to avoid a conflict of interest (For definition of conflict of interest see Rule 2.04 of The Rules of Professional Conduct of the LSUC).

Direct legal services

The services provided to a client that are directly related to a case.

Eligibility / referral / other service

For the purposes of Table 2.2.1, Resource Allocation includes the early service conducted by the clinic once a file is set up and information from the client has been entered into CIMS. It may include eligibility testing, referral and a limited level of service, and is calculated as the total number of hours in the reporting period for all client services where time docketed by all staff relating to the case in the period is less than two hours. It includes the activity of recording a potential clients information along with reason for their contact with the legal clinic.

Financial eligibility

Whether the client meets the clinic's financial eligibility guidelines.

Full-time equivalent (FTE)

The total FTE for a time period is calculated as the sum of hours allocated to all staff in the period as a proportion of the annual total hours available excluding estimated holiday, vacation and sick days (e.g. 1561 hrs per year for a full-time person). For example, if A works 200 hours or 200/1561 FTE (temporary worker), B works 780 hours or 780/1561 FTE (half time), and C works 1561 hours or 1561/1561 FTE (full time), the total FTE for all 3 staff for the year is (200+780+1561)/1561= 1.63 FTEs. For a 3-month period, it would be: 1.63/4=0.41FTE.

Initiative file

Work done or services provided in support of the initiative files (Initiative Type field in CIMS) such as; Public legal education/Outreach, Training, Community development, Policy advocacy, Law reform, Systemic advocacy, Partners/Network/Community Groups, LAO/Clinic Committee and Consultations, Inter-Clinic Groups, Memberships, Media/Communications, Governance, Professional development. These files may have a defined start and end date or be on-going with no finite end date.

Initiative file closing

Date when initiative is deemed to be completed (Close Date in CIMS).

Intake

This refers specifically to the process of initial data collection the activity of recording a potential clients information along with the reason for their contact with the legal clinic. The goal of this collection is to inform a clinics decision making about whether and/or how a clinic can best assist this person. It does not include any services provided to the client including drafting paperwork, assisting with forms or providing advice. It does include straight referrals. Intake for CIMS purposes just refers to the initial data collection.

Matter

A clients concern/issue/questions.

Referral

Service cannot be provided. Applicant given alternative source for assistance (this may be in combination with other services provided)

Salary

CIMS calculates the cost per case based on the personnel cost of docketed time, plus disbursements, minus recoveries. A salary or hourly rate must be entered and updated in the CIMS Salary Profile area to enable calculation of performance measures. Note: A value of $0.00 can be used (e.g. volunteers). Benefits should be included as applicable.

Service outcome definitions (to be used by clinic staff to determine case outcomes):

  • Successful: Meets all of the Clients Objectives
  • Partially successful: Meets some of the clients objectives
  • Unsuccessful: Does not meet clients objectives
  • Withdrawn/discontinued: Matter is withdrawn or legal action is discontinued by the clinic or client
  • Unknown: Clinic loses contact with client or outcome is unknown

Standard case

Is the service provided to a client where total docketed time by all staff is greater than two hours and the service relates to a specific situation or situations addressing a clients need.

Test case

Is service provided to a client, clients, or public interest matter where total docketed time is greater than two hours and the case is intended to have impact beyond the single case, such as public interest or law reform, and advances important public interests, transcends individual interests, and addresses a serious issue that fundamentally affects low-income Ontarians or disadvantaged communities whose perspective would be unlikely to come before the courts but for the clinics involvement.

2. Performance measures to be reported outside of CIMS

2.1 Measure #6: Service outcomes and client feedback

2.1.1 Efficiency measure

Category Satisfaction level (1-5) with case service as indicated by client
Timeliness of services provided 4.2
Ease of access to service 4
Responsive to needs 3.75
Treatment received from staff 4.1

2.1.2 Effectiveness measure

Average satisfaction level, on a scale of 1 (very dissatisfied) to 5 (very satisfied)

Category 2011 2012 2013
Client satisfied with amount of time to get service from the clinic 4 4.3 4.1
Found it easy to get service and assistance client was looking for 3 3.5 3.8
Client got what he/she needed 3.5 3 3.75
Client was treated with courtesy and respect by clinic staff 3.5 3.75 4.0

				Client feedback on service on a scale of 1 (very dissatisfied) to 5 (very satisfied). Chart shows timeliness as 4 in 2011, 4.3 in 2012, and 4.1 in 2013. 
				Ease of access to services was 3.5 in 2011, 3.75 in 2012, and 4 in 2013. 
				Responsive to needs was 3.5 in 2011, 3 in 2012, and 3.75 in 2013. 
				Treatment received from staff was 3 in 2011, 3.5 in 2012, and 3.8 in 2013.

Chart 10. Client feedback on service on a scale fo 1 (very dissatisfied) to 5 (very satisfied)

2.2 Measure #7: Complaints filed and founded

2.2.1 Sample data

Resolution stage of complaint # of complaints
Abandoned 12
Resolved by clinic 10
Resolved by board 2
Resolved by LAO 3
Not resolved To date 1
Total received 28
Total founded 3

2.2.2 Efficiency

Time to resolve complaint # of complaints %
Under 30 days 20 71%
Over 30 days 8 29%
Total complaints received 28  

				Pie chart for time required to resolve complaints in 2013. Chart shows 71% under 30 days and 29% over 30 days.

Chart 11. Time required to resolve comlplaints (2013)

2.2.3 Effectiveness

Category 2011 2012 2013
Complaints received 31 15 28
Complaints founded 5 2 3
% of complaints founded 16% 13% 11%

				Trend in founded complaints for 2011 to 2013. Chart shows complaints received as 31 in 2011, 15 in 2012, and 28 in 2013. 
				Complaints founded as 5 in 2011, 2 in 2012, and 3 in 2013. 
				Percent founded received as 16 in 2011, 15 in 2012, and 11 in 2013.

Chart 12. Trend in founded complaints (2011-2013)

Category 2011 2012 2013
Complaints received 31 15 28
Resolved in < 30 days 25 9 20
% resolved in < 30 days 81% 60% 71%

				Percentage of complaints resolved in less than 30 days for 2011 to 2013. Chart shows 81% in 2011, 60% in 2012, and 71% in 2013.

Chart 13. Percentage of complaints resolved in less than 30 days (2011-2013)

2.3 Governance scorecard

Governance scorecard

2.4 Quality indicators for non-case activity

Quality indicators for non-case activity (as recommended by ACLCO)

3. Related metrics included in CIMS but not part of performance measures reporting requirements

Client summary data for reporting period Total
Clients at beginning of period 1,200
New clients received during period 525
All clients served during period 1,725
Clients remaining at end of period 300
Client service completed during period 1,425
Initiative summary for reporting period Total
Active initiatives beginning of period 6
New initiatives commenced during period 4
Total initiatives open during period 10
Active initiatives remaining at end of period 3
Total initiatives2 closed during period 7
Initiative cases - Scope of impact Total %
Individual client/family 16 64%
Identifiable client group 2 8%
Community of interest: province-wide 2 8%
Community of interest: Beyond province-wide 4 16%
General population: Province-wide 6 24%
General population: Beyond province-wide 3 12%
Influence on decision-makers 6 24%
Total initiatives 25  
Case and client results #
# of clients (closed cases) 1725
Average cost per client $625
Initiative types conducted in period # of initiatives
Public legal education/outreach 2
Training 1
Community development 0
Policy advocacy 1
Law reform 1
Systemic advocacy 0
Partners/network/community groups 0
LAO/Clinic committee & consultations 0
Inter-clinic groups 1
Memberships 0
Media/communications 1
Governance 2
Professional development 0
Other 1
Total initiatives 11
Measure based on cases closed in fiscal year 2011 2012 2013
Average cost per client* $630 $670 $625
Average cases per client 1.7 1.9 1.8
Average days to close 26.9 27.9 23.6

				Average number of days to close case. Bar chart shows 27 in 2011, 28 in 2012, and 24 in 2013.

Chart 14. Average number of days to close case

Footnotes

Footnotes for section 2. Performance measures generated by CIMS

  1. Details re results achieved will be captured and will be available for detailed analysis as required
    Back to report

Footnotes for section 4. Related metrics included in CIMS but not part of Performance measures reporting requirements

  1. Details re complexity, results, and impact will be captured and will be available for detailed analysis as required
    Back to report

Personal information in this form is collected under the authority of section 84 of the Legal Aid Services Act and is used in the general administration of the payment of lawyers accounts including case management, application of block fees and tariff, discretion, reviews, disbursement authorization, expedite requests, late billing, hard cap, and recoveries; and is used in the panel management of lawyers including investigations, panel suspension, and panel removal.

Questions should be directed to the FIPPA coordinator, 40 Dundas Street West, Suite 200, Toronto, ON, M5G 2H1, 1-800-668-8258