Executive Summary

PyjamaHR’s AI Interview feature automates candidate screening with structured, unbiased video interviews and generates performance and quality scores for each candidate. This article explains how AI interview scoring works, how to interpret and use these scores, and provides actionable guidance for optimizing your hiring process, troubleshooting common issues, and ensuring a smooth candidate experience.

Detailed Overview

What Are AI Interview Performance and Quality Scores?

AI Interview Performance and Quality Scores are automated assessments generated by PyjamaHR’s AI Interviewer after a candidate completes a video interview. These scores are designed to help recruiters and hiring managers quickly evaluate candidate responses based on job-specific criteria, skills, and behavioral indicators.

Key Benefits:
- Unbiased, consistent evaluation: Every candidate is assessed using the same criteria, reducing human bias.
- Faster screening: Quickly identify top candidates without manually reviewing every interview.
- Scalable: Efficiently handle high applicant volumes, especially for roles with hundreds of applicants.

When to Use:
- For initial screening rounds where you need to filter large candidate pools.
- When you want a standardized, data-driven assessment before moving candidates to live interviews.
- To supplement human review with objective AI insights.

Integration with Other PyjamaHR Features:
- AI Interview scores are visible in candidate profiles and can be used alongside Resume Match Scores and other screening tools.
- Scores can be used to automate candidate progression through hiring stages.

Step-by-Step Guide: Using and Interpreting AI Interview Scores

1. Prerequisites

Ensure you have AI Interview credits (check under “My Plan” or purchase via the AI Store).
The AI Interview feature must be enabled for your job posting.
Candidates must have completed their AI interview.

2. Accessing AI Interview Scores

Navigate to the Job: Go to the relevant job from your PyjamaHR dashboard.
Open Candidate List: Click on the job to view all applicants.
Locate AI Interview Status: Candidates who have completed the AI interview will show a status such as “AI Interview Completed.”
View Scores: Click on a candidate’s profile. The AI Interview Score (sometimes labeled as “Performance Score” or “Quality Score”) will be displayed, often alongside a summary or transcript of the interview.

[Screenshot suggestion: Candidate profile with AI Interview Score highlighted]

3. Understanding the Score

Score Range: Typically, scores are presented as a percentage or a “High/Medium/Low” match.
Score Basis: The score is calculated based on:
Relevance and completeness of answers
Alignment with job-specific skills and criteria
Communication skills (for English-language interviews)
Behavioral and cognitive indicators (if configured)
Detailed Feedback: Some roles may include a breakdown of scores by skill or question.

4. Using Scores for Shortlisting

Bulk Actions: Filter candidates by score (e.g., shortlist all with “High” scores).
Compare View: Use the “Compare Candidates” feature to sort and export candidates by AI Interview Score.
Manual Review: For borderline cases, review the interview transcript or video alongside the score.

5. Advanced: Customizing AI Interview Criteria

When setting up the AI Interview, you can configure:
Screening questions
Required skills and experience levels
Behavioral or cognitive test components (if enabled)
The AI will use these configurations to tailor its scoring.

Advanced Usage & Best Practices

Power User Tips

Refine Screening Criteria: The more specific your job requirements and screening questions, the more accurate the AI scoring.
Combine with Resume Match: Use both Resume Match and AI Interview Scores for a holistic view.
Review Edge Cases: Always manually review candidates with “Medium” scores or those whose skills are hard to assess via AI.
Feedback Loops: If you notice the AI is consistently mis-scoring candidates, adjust your screening questions or criteria.

Do’s and Don’ts

Do:
- Ensure candidates have clear instructions and a stable internet connection.
- Encourage concise, relevant answers (long pauses or rambling can affect scoring).
- Use AI scores as a first filter, not the sole decision-maker.

Don’t:
- Rely solely on AI scores for final hiring decisions.
- Ignore candidate feedback about technical issues—these can impact scores.
- Forget to check for updates or improvements to the AI Interview system.

Troubleshooting & Common Issues

Common Error Messages & Solutions

Issue/Error	Solution
AI Interview score not visible / “Resume match is not working for this job”	Wait 10–15 minutes and refresh. If still missing, contact support with Job ID and candidate details.
“AI interviewer stopped responding” / blank screen	Ask candidate to rejoin using the same link. Check network and browser compatibility.
Candidate says “AI cut me off” or “didn’t give enough time to answer”	Advise concise answers. If persistent, report to support with candidate/job details for review.
“Exceeded maximum number of attempts”	Support can increase attempt limits. Contact support with candidate email and job details.
“AI says I am idle, please respond” even when speaking	Check microphone permissions. Try a different device or browser. If unresolved, support can review the interview.
“Interview incomplete” after candidate finishes	Support can manually mark as complete if the recording is valid.
Candidate feedback: “Questions too difficult for my experience level”	Review and adjust screening questions to match the intended candidate profile.
AI Interview stuck on “thinking” or “not starting”	Ask candidate to refresh, try a different browser, or rejoin later. If issue persists, escalate to support.
“No score generated” for a candidate	Ensure the candidate completed all questions. If so, support can backfill the score manually.

Recovery Procedures

Incomplete/Failed Interviews: Candidates can usually rejoin using the same link unless the maximum attempt limit is reached.
Technical Glitches: Support can reset attempts, extend expiry, or manually review interview recordings.
Score Discrepancies: If the AI score seems off, review the interview transcript and escalate to support for investigation.

When to Contact Support

Persistent technical issues affecting multiple candidates.
Missing or obviously incorrect scores.
Candidates unable to access or complete interviews after troubleshooting steps.

Comprehensive FAQ

1. How is the AI Interview Score calculated?
The score is based on candidate responses to pre-set questions, alignment with job-specific criteria, communication skills, and other configured parameters.

2. Where do I find the AI Interview Score?
In the candidate’s profile under the relevant job, after the interview is completed.

3. Can I customize the scoring criteria?
Yes, by configuring screening questions and required skills when setting up the AI Interview.

4. What if a candidate says the AI cut them off or didn’t give enough time?
Advise concise answers. If the issue persists, contact support with details.

5. Can candidates retake the AI interview?
Only if the interview is incomplete or if support increases the attempt limit. Completed interviews cannot be retaken.

6. What happens if a candidate’s interview is marked incomplete but they finished it?
Support can review the recording and manually mark it as complete.

7. How do I export or download AI Interview Scores?
Use the Compare Candidates view within a job to export candidate data, including scores, to Excel.

8. Can I filter candidates by AI Interview Score?
Yes, use the Compare View or filter options in the candidate list.

9. What if the AI Interview Score is missing or not updating?
Wait 10–15 minutes and refresh. If still missing, contact support with details.

10. Are AI Interview Scores available for all roles and languages?
Currently, AI interviews are only available in English.

11. How do I handle candidates with technical issues (mic, camera, network)?
Ask them to check their device settings, try a different browser or device, and rejoin using the same link.

12. Can I see a breakdown of the score by skill or question?
Some roles may include detailed breakdowns. If not visible, this feature may not be enabled for your account.

13. What if the AI interviewer is “stuck” or not responding?
Ask the candidate to refresh or rejoin. If unresolved, escalate to support.

14. How do I know if a candidate’s interview is truly complete?
If the candidate sees a “Thank you” or “Interview completed” message, it is complete. If in doubt, check the candidate’s profile or contact support.

15. Can I preview or test the AI interview before sending to candidates?
Currently, there is no preview mode, but you can create a test candidate and take the interview yourself.

Resume Match Score: Use alongside AI Interview Scores for a comprehensive candidate assessment.
Bulk Actions & Compare View: Filter, sort, and export candidates by AI Interview Score.
Custom Screening Questions: Tailor the AI Interview to your specific job requirements.
Candidate Communication: Use email templates to notify candidates of their interview status.
Troubleshooting Tools: Refer to the Troubleshooting section above for common issues.

Appendix: Real-World Scenarios and Examples

Scenario 1:
A recruiter notices that 120 out of 150 candidates are marked as “High” by the AI Interview Score.
Solution: Review and refine the screening questions and required skills for the job to improve score differentiation.
Scenario 2:
A candidate reports that the AI interviewer moved to the next question before they finished answering.
Solution: Advise candidates to keep answers concise and avoid long pauses. If the issue persists, escalate to support.
Scenario 3:
A hiring manager cannot see the AI Interview Score for a candidate.
Solution: Wait 10–15 minutes after interview completion, refresh the page, and check again. If still missing, contact support.

Summary

PyjamaHR’s AI Interview Performance and Quality Scores are powerful tools for efficient, unbiased candidate screening. By understanding how these scores work, following best practices, and proactively troubleshooting issues, you can maximize the value of AI-driven hiring and ensure a smooth experience for both your team and your candidates.

If you encounter any issues not covered here, please reach out to PyjamaHR Support with detailed information for prompt assistance.

Understanding AI Interview Performance and Quality Scores in PyjamaHR: Complete Guide, Troubleshooting, and Best Practices