Methodology & Technical Report
Comprehensive documentation of the Email Pen Test scoring algorithm and methodology
Overview
The Email Pen Test scoring system evaluates email security across three critical dimensions: Authentication, Threat Filter, and Inbox Protection. Each dimension contributes equally (33.3%) to the overall security score.
Overall Score Formula:
Overall Score = round((Authentication Score + Threat Filter Score + Inbox Protection Score) / 3)
1. Authentication Score (0-100)
Measures the effectiveness of email authentication mechanisms (DMARC, DKIM, SPF) in preventing unauthorized email delivery.
Calculation Method
- Starting score: 100 points
- Deduction: -20 points per authentication test that resulted in a bounce (authentication failure)
- Deduction: -10 points per test without delivery confirmation
- If no authentication tests exist: returns 50 points (neutral score)
Formula:
authScore = max(0, 100 - (totalDeductions / totalAuthTests))
where totalDeductions = (bounceCount × 20) + (noDeliveryCount × 10)
Example: If 2 out of 10 authentication tests bounced, and 1 had no delivery confirmation:
Deductions = (2 × 20) + (1 × 10) = 50
Score = max(0, 100 - (50 / 10)) = 95 points
2. Threat Filter Score (0-100)
Evaluates the accuracy of spam and threat filtering systems. Ideal behavior: risky emails go to spam, safe emails go to inbox. The scoring algorithm uses sophisticated protection-aware calculations that account for Google Workspace security features.
Risk Classification
Emails are classified as "risky" if they belong to these categories:
- Social Engineering
- Content Analysis
- Test cases containing "suspicious" or "mismatch" keywords
Protection Options & Mitigation
When a risky email reaches the inbox but has protection measures enabled, the penalty is reduced based on the protection's effectiveness. Protection options stack multiplicatively with diminishing returns:
Multiple protections stack: 1 - (1-w₁) × (1-w₂) × ... × (1-wₙ), capped at 90% total mitigation.
Penalty Structure
- Unprotected risky email in inbox: -50 points per occurrence (proportional to total tests)
- Protected risky email in inbox: -50 × (1 - mitigation) points (e.g., 75% protection = -12.5 points)
- Risky email in spam: 0 points (ideal outcome)
- Risky email not received: -5 points per occurrence (blocked, but uncertainty penalty)
- Safe email in inbox: 0 points (ideal outcome)
- Safe email in spam: -20 points per occurrence (false positive, proportional to total tests)
- Safe email not received: -10 points per occurrence (unusual, penalized)
- No placement data: -10 points per occurrence (uncertainty penalty)
Formula:
filteringScore = max(0, 100 - averagePenalty)
where averagePenalty = totalPenalty / totalTests
For each risky email in inbox:
penalty = 50 × (1 - protectionMitigation)
protectionMitigation = 1 - ∏(1 - protectionWeight)
capped at 0.90 (90% maximum mitigation)
Example 1 (Unprotected): Out of 20 test emails:
• 2 unprotected risky emails in inbox: 2 × 50 = 100 penalty
• 1 safe email in spam: 1 × 20 = 20 penalty
• Average penalty = 120 / 20 = 6 points
Score = 100 - 6 = 94 points
Example 2 (Protected): Same scenario, but both risky emails have URL Protection (75% mitigation):
• 2 protected risky emails in inbox: 2 × (50 × 0.25) = 25 penalty
• 1 safe email in spam: 1 × 20 = 20 penalty
• Average penalty = 45 / 20 = 2.25 points
Score = 100 - 2.25 = 98 points
3. Inbox Protection Score (0-100)
Measures the actual user risk from risky emails that reached inboxes, accounting for protection measures that reduce exposure. This score reflects real-world security posture: a risky email with strong protection is less dangerous than an unprotected one.
Calculation Method
For each risky email, exposure is calculated based on placement and protection:
- Risky email in spam or not received: 0.0 exposure (no user risk)
- Unprotected risky email in inbox: 1.0 exposure (full user risk)
- Protected risky email in inbox: 1.0 - (protectionMitigation × 0.9) exposure
- Risky email with no placement data: 0.5 exposure (uncertainty penalty)
The total exposure ratio is calculated as: totalExposure / totalRiskyEmails
Score = 100 × (1 - exposureRatio), clamped between 0 and 100
Formula:
For each risky email:
if (spam || not_received) exposure = 0.0
else if (inbox) exposure = 1.0 - (protectionMitigation × 0.9)
else exposure = 0.5
exposureRatio = Σ(exposure) / totalRiskyEmails
exposureScore = round(100 × (1 - exposureRatio))
Example 1 (Unprotected): Out of 8 risky emails:
• 2 unprotected in inbox: 2 × 1.0 = 2.0 exposure
• 6 in spam: 6 × 0.0 = 0.0 exposure
Exposure Ratio = 2.0 / 8 = 0.25 (25%)
Score = 100 × (1 - 0.25) = 75 points
Example 2 (Protected): Same scenario, but both inbox emails have URL Protection (75% mitigation):
• 2 protected in inbox: 2 × (1.0 - 0.75 × 0.9) = 2 × 0.325 = 0.65 exposure
• 6 in spam: 6 × 0.0 = 0.0 exposure
Exposure Ratio = 0.65 / 8 = 0.081 (8.1%)
Score = 100 × (1 - 0.081) = 92 points
Grade Assignment
The overall score (0-100) is mapped to letter grades as follows:
| Grade | Score Range | Interpretation |
|---|---|---|
| A+ | 97-100 | Exceptional security posture |
| A | 93-96 | Excellent security posture |
| A- | 90-92 | Very good security posture |
| B+ | 87-89 | Good security posture |
| B | 83-86 | Above average security |
| B- | 80-82 | Average security |
| C+ | 77-79 | Below average, improvements needed |
| C | 73-76 | Needs significant improvement |
| C- | 70-72 | Poor security posture |
| D+ | 67-69 | Critical gaps present |
| D | 63-66 | Major security vulnerabilities |
| D- | 60-62 | Severe security issues |
| F | 0-59 | Immediate action required |
Gap Analysis
The system automatically identifies security gaps based on test results:
High Impact Gaps
- Authentication Failures: Test emails that bounced due to authentication issues
- Risky Emails in Inbox: Social engineering or suspicious emails that reached users' inboxes
Medium Impact Gaps
- False Positives: Safe emails incorrectly filtered to spam
Low Impact Gaps
- Incomplete Results: Test emails without placement confirmation (>10% of total)
Test Methodology
Email Pen Test sends 20 safe, authorized test emails covering:
Authentication Tests
Validates DMARC, DKIM, and SPF configuration through various authentication scenarios.
Threat Filter Tests
Tests spam detection, social engineering detection, and content analysis capabilities.
Social Engineering
Simulates common phishing patterns and social engineering tactics (safely marked as tests).
Content Analysis
Tests detection of suspicious content patterns, link wrapping, and evasion techniques.
Important: All test emails are clearly marked as security tests and are safe to receive. They are designed to test your email security configuration, not to bypass it.
Limitations & Considerations
- Scores are based on a snapshot in time and may vary based on email provider filtering updates
- Test results reflect Gmail/Google Workspace filtering behavior, which may differ from other providers
- Placement results depend on user interaction and may require manual verification
- The scoring algorithm prioritizes preventing risky emails from reaching inboxes over minimizing false positives
- Authentication scores assume proper DMARC/DKIM/SPF configuration; misconfiguration will result in lower scores
Technical Implementation
The scoring algorithm is implemented in TypeScript and runs server-side. Key technical details:
- All scores are rounded to the nearest integer
- Minimum score is clamped to 0 (no negative scores)
- Maximum score is 100 (perfect security posture)
- Subscores are calculated independently before averaging
- Gap analysis runs after score calculation to identify specific issues
Questions or Feedback?
If you have questions about the methodology or would like to provide feedback, please contact us.
Put Our Methodology to the Test
Experience our comprehensive scoring system firsthand. Start your free email security test today.