How Do You Know Your School Safety Program Is Working? Building a Simple Safety Scorecard

School districts invest significant time, money, and personnel into school safety. Emergency plans are written. Drills are conducted. Employees attend training. Behavioral threat assessment teams meet. Cameras are installed. Doors are checked. Communication systems are purchased. Safety assessments are completed. All of those things can be important.
But there is a question I believe school and district leaders need to ask more often:
How do we know any of it is actually working?
That question is harder to answer than it might seem. The easiest approach is to count activities. How many drills did we conduct? What percentage of staff completed training? How many threat assessments were completed? When was the Emergency Operations Plan last updated? Those numbers have value, particularly for determining compliance.
But they do not necessarily tell us whether the organization is becoming more prepared. A district can conduct every required drill and learn nothing from them. One hundred percent of employees can complete training without understanding what to do during an emergency.
A Behavioral Threat Assessment and Management team can complete an assessment without effectively managing the case afterward. An Emergency Operations Plan can be updated annually while containing procedures that have never been tested. Activity is not the same thing as capability.
If we want to understand whether a school safety program is working, we need a better way to measure it. That does not require an enormous data system or a complicated formula that supposedly tells us a school is "87% safe." Instead, districts can build a relatively simple school safety scorecard designed around a more useful question:
Are the systems we have put in place functioning, and are they improving over time?
At School Safety Solutions, this question is central to how we approach school safety assessment. Rather than viewing an assessment as a one-time inspection that produces a report, our process is designed to help districts establish what should be in place, determine how consistently those systems are being implemented, validate them in practice, and then create a structure for monitoring improvement over time.
The scorecard is not the beginning of that process. It is one of the outcomes.
Don't Start with the Scorecard
This may sound strange in an article about building a scorecard, but the scorecard should not be the starting point. In our work with schools and districts, we begin with an evidence-based assessment of the safety program itself. Before measuring performance, a district needs to determine what systems and processes should reasonably be in place.
That assessment should go well beyond the traditional security audit. School safety is not just cameras, doors, fences, and law enforcement. A comprehensive review should look across the organization. For example, our district-level assessment process examines areas such as: Safety and security. Student support. Maintenance and operations. Technology. Health services.
Transportation. Communications. Athletics and special events. Within those areas, the questions become much more operational. Does the district have an Emergency Operations Plan aligned with accepted emergency-management practices? Are safety drills required? How are exterior doors monitored? How are visitors, contractors, and vendors admitted?
Can emergency announcements be heard throughout campuses? Are fire alarms and other emergency systems regularly tested? Can employees report safety concerns? Are campus maps current and available? Is life-safety equipment inspected? Are transportation personnel trained in emergency procedures? Are emergency messages pre-drafted? Is there a primary and backup spokesperson?
How are stadium access points monitored? How are behavioral threat assessment and student-support processes implemented? These questions establish the system expectations. The initial assessment should help district leaders determine:
What is already functioning well?
What is missing?
What is inconsistent?
What requires greater attention?
From there, we can work with the district to identify high-level recommendations about the activities and strategies that need to be developed, strengthened, or standardized. Only then should we decide what belongs on the scorecard.
Move From District Expectations to Campus Implementation
The next step is critical. A district may have a strong procedure on paper. That does not necessarily mean the procedure is happening consistently across campuses. This is where we move from the district-level assessment to a customized procedural alignment checklist. Take emergency planning as an example.
At the district level, the assessment may ask whether campuses maintain Emergency Operations Plans. At the campus level, the questions become more specific: Has the EOP been updated for the new school year? Has the campus Incident Command structure been updated? Are campus maps and floor plans current? Have contact lists been updated?
Have staff identified as members of the safety team reviewed the plan? Have administrators and staff discussed how emergency actions will actually be implemented? Those questions move us from:
"Do we have a plan?"
to:
"Is the plan operational?"
The same principle applies to Lockdown. The district may have adopted a Lockdown protocol. The campus checklist should go further. Have administrators identified areas that are difficult or impossible to lock down effectively? Has a specific plan been developed for those locations? Have appropriate gathering areas been identified for individuals who may need to self-evacuate?
Do staff understand those options? That is a much better measure of preparedness than simply asking whether Lockdown appears in the EOP.
One important part of our process is that the campus checklist does not have to be identical for every district. The evidence-based district assessment helps determine what should be emphasized. Those findings can then be used to customize beginning-of-year, mid-year, end-of-year, and ongoing procedural checks around the district's actual needs.
Ask Principals What the Checklist Cannot Tell You
Not everything worth knowing fits neatly into a yes/no question. That is why our campus assessment process also includes structured conversations with principals. Ask:
What are the most challenging safety and security issues on your campus?
What are the biggest barriers to resolving them?
What does your campus do particularly well?
What needs the most improvement?
Where do staff need additional training?
Are there areas you want the district safety team to examine more closely?
Then ask similar questions about student support and health services. This does something a checklist cannot. It provides context. Perhaps a campus repeatedly reports an exterior-door issue. The principal may explain that the real problem is not the door---it is a staffing and supervision challenge during arrival.
Another campus may have strong emergency procedures but significant concerns around a particular athletic facility. Another may identify substitute preparedness as its greatest vulnerability.
Data tell us where to look. Conversations often tell us why the problem exists.
Both are necessary.
Validate What You Find On Site
Once the district-level assessment and campus procedural review are complete, the next step in our process is validation. Go to the school. Walk the campus. Observe a drill. Look at access control. Talk to staff. Test assumptions. There is often a difference between:
"We have a procedure."
and
"This is what actually happens."
That is why we incorporate onsite walkthroughs and practical assessments rather than relying entirely on documents and self-reported information. Consider access control. The written procedure may require visitors to enter through a designated entrance. During the walkthrough, ask:
Can someone bypass that entrance? Are secondary exterior doors secured? Are doors being propped open? Are gates functioning? Does signage direct visitors appropriately? Do employees understand what to do when they encounter an unidentified visitor? Does the visitor-management system operate as expected? The same approach should be used with drills.
Do not simply review the drill log. Observe the exercise. Did everyone hear the notification? Did staff understand the protocol? What happened in the cafeteria? What happened outside? Were students accounted for? Did administrators understand their roles? Were there communication problems? What happened afterward? This is where the assessment becomes real.
Build the Scorecard From What You Learn
Now we can build the scorecard. The scorecard should not contain every question from the district and campus assessments. That would make it unusable. Instead, leadership should see a small number of indicators that tell them whether the most important systems are functioning.
In our process, those measures should be informed by what we learned during the district assessment, campus procedural review, principal conversations, walkthroughs, and exercises---not selected simply because they are easy to count. I would begin with eight to ten areas.
1. Corrective-Action Completion
This may be one of the most important measures in the entire program. Every drill, exercise, assessment, walkthrough, or real incident can identify problems. The question is:
Do we fix them?
FEMA's Homeland Security Exercise and Evaluation Program emphasizes improvement planning and the tracking of corrective actions as part of building and sustaining preparedness. A district could track:
Percentage of corrective actions completed by the assigned deadline.
But I would add another measure:
Number of high-priority corrective actions that are overdue.
If a campus identified poor PA coverage six months ago and the problem remains unresolved, leadership should know. Every meaningful corrective action should have: A finding. A specific action. An owner. A deadline. A method for verifying completion. Finding a problem is valuable.
Fixing it is the measure that matters.
2. Recurring Vulnerabilities
Now take corrective actions one step further. What problems keep coming back? Perhaps multiple campuses repeatedly identify: Student accountability problems. Exterior-door concerns. PA coverage issues. Confusion about Incident Command. Substitute preparedness. Radio limitations. These patterns are incredibly valuable.
A vulnerability appearing at one school may be a campus problem. The same vulnerability appearing at six schools is probably a district system problem. The scorecard should identify recurring findings so leaders can see where district-level intervention, funding, policy changes, or training may be needed.
3. Exercise Performance
Schools already conduct a significant number of emergency drills. The question should not simply be whether they occurred. Ask:
What did we learn?
A useful scorecard might track: Percentage of required drills completed. Percentage reviewed afterward. Number of significant findings. Percentage of findings converted to corrective actions. Percentage of previous findings successfully re-tested.
Federal school emergency-planning guidance emphasizes using exercises to validate plans, clarify roles, assess resources, identify gaps, and strengthen preparedness. That is the real value of an exercise. A drill that produces a meaningful improvement is more valuable than one that simply produces another completed form.
4. Staff Knowledge
Training completion should still be measured. But pair it with demonstrated knowledge. For example:
Training completion: 97%
Staff sampled who correctly understand core emergency actions: 84%
That difference tells leadership something important. It also helps identify where additional training is needed. Break the results down by role. Teachers. Administrators. Paraprofessionals. Transportation. Substitutes. Coaches. Office staff. Custodians. The districtwide average can sometimes hide the group most likely to struggle during an emergency.
The purpose is not to test employees for punishment. It is to test whether the training program is working.
5. Behavioral Threat Assessment Case Management
Behavioral Threat Assessment and Management provides another good example of why activity counts can be misleading. Consider this statistic:
124 behavioral threat assessments completed.
Is that good? Bad? High? Low? We really do not know. Instead, examine the quality of the process after assessment. Among cases requiring ongoing management: What percentage have a documented management plan? What percentage have an identified case owner? What percentage are being reviewed according to established timelines?
What percentage of assigned interventions have been implemented? Are any cases overdue for review? Those measures tell us whether assessment is translating into management. The data should be aggregated to protect student privacy. District leadership does not need confidential case details on a dashboard. It does need to know whether the system is functioning.
6. Emergency Plan Maintenance
Many districts measure whether plans are updated annually. Keep that measure. But add:
What caused the plan to change?
If exercises repeatedly identify problems but the EOP never changes, something is disconnected. Federal guidance treats plan maintenance as a continuous process involving training, exercises, review, revision, and improvement. A useful scorecard might track: Percentage of campus EOPs reviewed on schedule. Percentage of campus Incident Command charts current.
Number of significant procedural changes resulting from exercises or incidents. Number of annexes requiring revision. The objective is not constant rewriting. Sometimes an exercise validates that the existing plan works. The important thing is that there is a connection between:
Experience → Evaluation → Plan Improvement
7. Communication Performance
Emergency communication is another area where ownership of technology can be mistaken for capability. The question is not:
"Do we have a mass notification system?"
Ask:
"Does it work under the conditions our plan assumes?"
Measure: Can PA announcements be heard throughout the campus? Can staff receive emergency notifications? Can administrators initiate the system? Are backup systems available? Can communications personnel quickly access pre-drafted emergency messages? Is there a primary and secondary spokesperson?
How quickly can the district produce an accurate family message during an exercise? Communication can be tested. If it can be tested, it can be measured.
8. Access Control and Physical Systems
Physical-security findings should also appear on the scorecard---but focus on performance rather than equipment counts. Instead of:
Number of cameras installed
consider:
Percentage of identified camera-coverage deficiencies corrected.
Instead of:
Number of exterior doors
measure:
Percentage of access-control deficiencies resolved within established timelines.
Instead of:
AEDs are present
ask whether required life-safety inspections are occurring and deficiencies are being corrected. This keeps the focus on whether systems function rather than how much equipment has been purchased.
Customize Monitoring Throughout the Year
Once the initial onsite assessment is complete, another important part of our approach is using what we learned to customize ongoing campus monitoring. The initial assessment establishes the baseline. The goal after that is not simply to return a year later and conduct the same assessment again.
Instead, the findings should help determine what needs to be monitored throughout the school year. I like dividing this across the year.
First Nine Weeks: Establish the Baseline
Focus on foundational readiness. Emergency Operations Plans. Incident Command assignments. Contact lists. Maps. Staff training. Access control. Emergency equipment. Communication systems. Go-kits. Campus safety teams.
Second Nine Weeks: Test Implementation
Now ask whether systems are functioning. What did early drills identify? Are corrective actions being completed? Are exterior doors being managed as expected? Can staff explain emergency protocols? Are substitute procedures working? Are threat-management cases receiving required follow-up?
Third Nine Weeks: Test More Complex Capabilities
Move toward issues such as: Incident Command. Reunification. Transportation. Emergency communication. Athletics and special events. Community-partner coordination. Backup systems.
Fourth Nine Weeks: Evaluate and Improve
Finish by looking backward and forward. Which corrective actions remain open? Which vulnerabilities recurred? What did exercises teach us? What needs to change in the EOP? What training needs emerged? What equipment or capital improvements should be considered? What should become a priority next year? Now the assessment is not something that happened once.
It has become a management process.
Keep the Leadership Scorecard Simple
The superintendent and cabinet do not need 150 safety measures every quarter. Give them the indicators that matter. A practical scorecard might look like this:
Corrective actions completed on time — 86%, Improving. Action needed: Review overdue high-priority items.
Recurring vulnerabilities — 7, Improving. Action needed: Focus on accountability and PA coverage.
Staff protocol knowledge — 89%, Stable. Action needed: Target substitutes and transportation.
BTA management reviews on time — 94%, Improving. Action needed: Follow up on overdue cases.
Campus EOPs current — 100%, Stable. Action needed: Continue quarterly verification.
Communication tests successful — 92%, Improving. Action needed: Correct exterior-area coverage.
Access-control findings resolved — 81%, Declining. Action needed: Facilities review required.
Exercise findings re-tested — 76%, Improving. Action needed: Build findings into spring exercises.
Those numbers are illustrative, but the structure is what matters. The scorecard should answer:
Where are we?
Are we improving?
Where does leadership need to intervene?
Don't Punish Campuses for Finding Problems
There is one cultural issue that can make or break this process. If principals believe red or yellow findings make them look bad, the quality of your data will quickly deteriorate. The objective is not to make every campus green. The objective is to find vulnerabilities before an emergency finds them for you.
A principal who identifies that the playground cannot hear the PA system has done something valuable. Now the district can fix it. A campus that discovers an accountability weakness during an evacuation drill has learned something important. Now it can improve. The problem is not finding vulnerabilities.
The problem is finding the same vulnerabilities repeatedly and failing to address them.
The scorecard should support:
Identify → Prioritize → Correct → Verify
Not:
Identify → Blame
That philosophy is important to how we approach assessments at School Safety Solutions. The assessment should help district and campus leaders improve their systems---not simply hand them a list of deficiencies.
The Most Important Measure: Are We Getting Better?
Ultimately, this is the question the scorecard should help answer. Not: How many drills did we conduct? How many people attended training? How many cameras did we buy? How many plans did we update? Those measures can still be useful. But they should feed a larger conversation. Are staff more knowledgeable than they were last year? Are corrective actions being completed faster?
Are recurring vulnerabilities decreasing? Are exercises becoming more effective? Are threat-management cases receiving consistent follow-up? Are communication systems performing better? Are plans changing based on what we learn? Are campuses implementing district procedures consistently?
That is how we begin moving from measuring activity to measuring capability.
At School Safety Solutions, this is the larger goal behind our assessment process. We do not believe a meaningful school safety assessment should end with a report identifying what is wrong.
The assessment should create the foundation for an ongoing process:
Evidence-Based Assessment → Prioritize → Operationalize → Validate → Monitor → Improve → Reassess
We begin by evaluating district-level systems against evidence-based practices. Those findings help identify high-level priorities and recommendations. We then translate those expectations into practical campus-level procedures and customized checklists. We talk with campus leaders. We walk buildings. We assess access control.
We observe drills and evaluate how procedures perform in practice. Those findings can then inform customized monitoring throughout the school year and a small set of meaningful indicators for district leadership. The result is not simply another assessment report. It is a way to help leaders continually answer:
What should be happening?
Is it actually happening?
How do we know?
What have we learned?
What needs to change?
Did the change work?
Because ultimately, the question is not whether your district can demonstrate that it does school safety activities. The better question is:
Can you demonstrate that those activities are making your organization more prepared?
That is a much harder standard. But it is also a much more meaningful measure of whether your school safety program is actually working.
Want help building an evidence-based safety scorecard for your district? Explore our school safety services or reach out to talk through next steps.
School Safety Solutions helps K-12 leaders strengthen prevention, protection, and preparedness through practical assessment, training, tabletop exercises, Standard Response Protocol implementation, behavioral threat assessment support, and reunification planning. If your district wants a calm, working review of its active shooter protocol and broader emergency readiness, visit School Safety Solutions or call 512-387-6729 to talk through next steps.




Comments