Evaluation Period: Managing the Judging Phase
Entries are in, judges are assigned. Now manage the evaluation period smoothly: monitor scoring progress, handle issues, and finalize results that everyone trusts.
The Evaluation Period Mindset
Your job during the evaluation period is not to score entries yourself. Your job is to keep judges on track, resolve issues quickly, and ensure the process runs smoothly. Think of yourself as the operations manager, not a judge.
What Program Managers Should Avoid
- ✗Influencing judge scores or opinions
- ✗Waiting until the last day to check judge progress
- ✗Manually recalculating scores in spreadsheets
- ✗Ignoring scoring outliers or inconsistencies
What Great Program Managers Do
- Send weekly progress updates to judges
- Monitor completion rates and follow up with slow judges
- Flag and investigate significant scoring discrepancies
- Be available to answer judge questions promptly
Set Expectations Early
Week-by-Week Evaluation Management
A typical evaluation period runs 2 to 3 weeks. Here is how to manage each phase to keep scoring on track.
Week 1: Launch and Monitor
Open scoring and ensure all judges can access their assigned entries.
Week 2: Progress Check and Quality Review
Mid-point review. Most judges should be 50% or more complete.
Week 3: Final Scoring and Results
Close scoring, review final results, and prepare for announcements.
Common Evaluation Issues and Solutions
These problems come up in nearly every award program. Here is how to handle them and keep the process moving.
"A judge is not responding to reminders"
Solution: After two unanswered reminders, call them directly. If they cannot complete, redistribute their entries to your backup judge. Do not let one judge delay the entire program.
Prevention: Set clear expectations upfront and recruit 1 to 2 backup judges.
"Two judges scored the same entry very differently"
Solution: If scores differ by more than 3 points on a 10-point scale, investigate. Ask both judges to explain their rationale. If needed, bring in a third judge as a tiebreaker.
Prevention: Run a calibration session before judging begins with a sample entry.
"A judge has a conflict of interest with an entry"
Solution: Remove the entry from that judge's queue and reassign it. If the judge flagged the conflict themselves, thank them for their integrity.
Prevention: Share the entrant list with judges before scoring opens and ask them to flag any conflicts.
"An entrant claims their entry was incomplete"
Solution: Check the entry record. If a technical issue caused the problem, allow a reentry within 24 hours. If the entrant simply missed a deadline, the rules apply equally to everyone.
Prevention: Send entry confirmations with a summary of what was received.
"The results are very close between two entries"
Solution: If entries are within 0.5 points, convene the full panel for a deliberation discussion. Let the judges collectively decide on the winner with a recorded vote.
Prevention: Document your tiebreaker procedure in advance so you are not making rules on the fly.
Signs of a Well-Run Evaluation
You will know your evaluation period was successful when:
"The judging process was seamless. I just logged in, scored, and was done."
Judges appreciated the efficiency and clarity of the process.
"I trust the results because the criteria were clear from the start."
Entrants accept the results because the process was transparent.
"Can I judge again next year? I enjoyed seeing the quality of work in our industry."
Judges want to return, which builds continuity for your program.