Detecting authorship gradients in the Pauline corpus through one-class neural classification with Ancient Greek BERT embeddings.
Can neural language models detect what two centuries of scholarship have debated? We use zero-shot embeddings from Ancient Greek BERT to measure semantic distance between undisputed Pauline texts and letters of contested authorship.
Scholars have long debated which New Testament letters attributed to Paul were genuinely written by him. Traditional stylometry (word-frequency analysis) struggles because the disputed letters are stylistically very close to authentic Paul. Our approach asks: do modern neural embeddings — which capture deep semantic and syntactic patterns — reveal an authorship gradient that correlates with scholarly consensus?
| Text | Scholarly Status | Rejection Rate |
|---|---|---|
| Undisputed Paul | Authentic | 0% |
| Colossians | Contested | ~40% |
| 2 Thessalonians | Contested | ~50% |
| Ephesians | Contested | ~60% |
| 1 Timothy | Likely pseudepigraphal | ~80% |
| 2 Timothy | Likely pseudepigraphal | ~80% |
| Titus | Likely pseudepigraphal | ~80% |
| Hebrews | Non-Pauline (consensus) | 100% |
A one-class classification approach that avoids circular reasoning by building the baseline exclusively from undisputed texts.
The baseline centroid and variance are computed using only undisputed Pauline texts (Romans, 1–2 Corinthians, Galatians, Philippians, 1 Thessalonians, Philemon). Disputed texts are then measured against this baseline without influencing it — avoiding the circular reasoning that plagues traditional multi-class approaches.
Cosine distance with mean pooling, z-score normalised by intra-Pauline variance:
Cosine distance is the literature standard for sentence embeddings (Reimers & Gurevych 2019), measuring directional similarity independent of magnitude.
pranaydeeps/Ancient-Greek-BERTSemantic distance from the Pauline baseline plotted against scholarly rejection rate. The positive trend (ρ = 0.704) shows that texts more frequently rejected by scholars tend to diverge further in neural embedding space.
Each point represents a disputed text. The x-axis shows the percentage of scholars who reject Pauline authorship; the y-axis shows how far the text's embedding centroid sits from the undisputed Pauline centroid (in standard deviations). The dashed trend line captures the positive correlation. Hebrews and 1 Timothy — coloured in rose — stand out as the clearest outliers, while 2 Thessalonians and Ephesians (green) sit close to or below the Pauline baseline.
Distance from Paul (in σ units) with 95% confidence intervals and significance testing. Only Hebrews and 1 Timothy reach statistical significance.
| Text | N chunks | Distance (σ) | 95% CI | P-Value | Cohen's d | Verdict |
|---|---|---|---|---|---|---|
| Paul (baseline) | 314 | 0.00 | — | — | — | Baseline |
| Colossians | 20 | 0.24 | [−0.20, 0.69] | 0.261 | 0.25 | Indistinguishable |
| 2 Thessalonians | 10 | −0.22 | [−0.70, 0.26] | 0.320 | −0.26 | Indistinguishable |
| Ephesians | 31 | −0.03 | [−0.39, 0.33] | 0.868 | −0.03 | Indistinguishable |
| 1 Timothy | 20 | 0.79 | [0.39, 1.20] | 0.0007 | 0.84 | Significant ✦ |
| 2 Timothy | 15 | 0.11 | [−0.61, 0.83] | 0.748 | 0.09 | Indistinguishable |
| Titus | 8 | 1.11 | [−0.10, 2.32] | 0.067 | 0.89 | Marginal |
| Hebrews | 65 | 1.21 | [0.75, 1.67] | <0.0001 | 0.81 | Significant ✦ |
What percentage of each text's chunks fall outside Pauline percentile thresholds? Texts with true authorial divergence should show many chunks in the tails.
By definition, 25% of Pauline chunks exceed P75, 10% exceed P90, and 5% exceed P95. Values substantially above these baselines indicate systematic divergence. Hebrews shows the most extreme pattern: 58.5% of its chunks exceed P75 and 38.5% exceed P95 — nearly eight times the expected rate. 1 Timothy follows with 65% above P75. In contrast, 2 Thessalonians shows fewer extreme chunks than expected, with 0% above P90 — consistent with its negative mean distance.
PCA projection of BERT embedding chunks reveals clear geometric separation between Pauline and non-Pauline texts in the latent space.
The scatter plot shows embedding chunks projected via PCA. Pauline texts (indigo dots) form a tight cluster in the positive PC1 region. Hebrews (rose triangles) separates cleanly into negative PC1, confirming neural divergence. Colossians (amber diamonds) occupies a distinct position on PC2, intermediate between Paul and Hebrews — consistent with its contested status. This geometric structure is invisible to classic word-frequency methods.
PCA on most-frequent-word z-scores (100 MFW) shows high lexical similarity across all texts — classic methods cannot distinguish disputed letters from Paul.
Classic stylometry relies on surface-level word frequencies. All Pauline and pseudo-Pauline texts cluster tightly in PCA space (blue box) because the disputed authors successfully mimicked Paul's vocabulary. Only Hebrews shows slight separation on PC2. Colossians (amber) is the most distant on PC1, yet the total variance explained is extremely low (4.98% + 3.72%), meaning even these small differences may be noise. This motivates the neural approach, which captures deeper semantic and syntactic patterns invisible to word-counting methods.
The neural probe detects divergence where classic stylometry does not — a dissociation that validates the complementary value of deep embeddings.
Classic stylometry (amber bars) shows uniformly high lexical overlap with Paul across all texts, ranging from 0.82 (Hebrews) to 0.97 (2 Thessalonians). The neural distance measure (indigo bars) reveals a starkly different picture: Hebrews (1.21σ), Titus (1.11σ), and 1 Timothy (0.79σ) stand well above the baseline, while 2 Thessalonians (−0.22σ) and Ephesians (−0.03σ) remain close or below. This dissociation demonstrates that neural embeddings capture information about authorship that surface-level word frequencies miss entirely.
What the neural probe reveals about the Pauline corpus.
Semantic distance correlates with scholarly rejection rates (ρ = 0.704). Texts more frequently rejected as non-Pauline show greater neural divergence, producing a monotonic trend that approaches conventional significance (p = 0.077) with only 7 data points.
Hebrews — universally regarded as non-Pauline — shows the strongest divergence (1.21σ, p < 0.0001, d = 0.81), with 38.5% of chunks outside the 95th percentile. This validates the method's ability to detect genuine authorial differences.
1 Timothy shows clear divergence (d = 0.84, p = 0.0007), while 2 Timothy remains indistinguishable (d = 0.09, p = 0.748). This may reflect different compositional histories or varying degrees of authentic material embedded in each letter — a finding consistent with some recent scholarship.
2 Thessalonians (−0.22σ) and Ephesians (−0.03σ) show negative distances, clustering closer to Paul than average. This is consistent with either authentic authorship or high-quality stylistic mimicry that captures deep semantic patterns beyond mere word-frequency imitation.