How to fix Error 13189 on Windows Server 2025 / Windows 11

by

Troubleshooting Guide: SQL Server Error 13189 — Invalid Key Exchange Key Size After Decryption

Why This Error Happens

This error occurs when SQL Server 2025 attempts to decrypt the key exchange key (KEK) using a certificate’s private key, but the resulting decrypted value does not match the expected size for the algorithm in use. This typically happens due to a corrupted or mismatched certificate, an incompatible encryption algorithm negotiated between endpoints (common in Always On Availability Groups or Database Mirroring setups), or a certificate that was improperly restored/re-issued with a different key length. It can also surface after a Windows Server 2025 security update changes default cryptographic provider behavior (CNG vs. legacy CAPI).

Step-by-Step Resolution Guide

Step 1: Verify the Certificate Integrity on Both Endpoints

  • Run SELECT * FROM sys.certificates WHERE name = 'YourCertName'; on both the principal and mirror/replica instances.
  • Confirm the pvt_key_encryption_type_desc matches on both servers (should be consistent, e.g., ENCRYPTED_BY_MASTER_KEY).
  • Check the certificate expiration date and thumbprint using SELECT thumbprint FROM sys.certificates; to ensure they are identical on both sides.

Step 2: Re-Export and Re-Import the Certificate with Matching Key Length

  • Backup the certificate on the source instance using:
    BACKUP CERTIFICATE YourCertName TO FILE = 'C:\Certs\YourCert.cer' WITH PRIVATE KEY (FILE = 'C:\Certs\YourCert.pvk', ENCRYPTION BY PASSWORD = 'StrongPassword123!');
  • Drop the existing certificate on the destination instance using DROP CERTIFICATE YourCertName;.
  • Re-create it using CREATE CERTIFICATE YourCertName FROM FILE = 'C:\Certs\YourCert.cer' WITH PRIVATE KEY (FILE = 'C:\Certs\YourCert.pvk', DECRYPTION BY PASSWORD = 'StrongPassword123!');.
  • Ensure both certificates use the same key length (2048-bit minimum is recommended for SQL Server 2025).

Step 3: Check Cryptographic Provider Compatibility on Windows Server 2025

  • Open certlm.msc and navigate to the certificate under Personal > Certificates.
  • Right-click the certificate, go to Properties, and confirm the Cryptographic Service Provider (CSP) is set to a compatible provider (e.g., “Microsoft Enhanced RSA and AES Cryptographic Provider”).
  • If the provider shows as a CNG-only provider and SQL Server expects legacy CAPI, re-export the certificate with the legacy CSP option enabled via certutil -csp "Microsoft Enhanced RSA and AES Cryptographic Provider" -importpfx YourCert.pfx.

Step 4: Restart the SQL Server Service and Validate Endpoint Configuration

  • Restart the SQL Server service via services.msc or run Restart-Service MSSQLSERVER in PowerShell (run as Administrator).
  • Re-check the mirroring/AG endpoint state using SELECT * FROM sys.database_mirroring_endpoints; to confirm the state is STARTED.
  • Review the SQL Server Error Log (C:\Program Files\Microsoft SQL Server\MSSQL17.MSSQLSERVER\MSSQL\Log\ERRORLOG) for confirmation that Error 13189 no longer appears.

Alternative or Verification Steps

  • To verify the fix, attempt to re-establish the Always On Availability Group session or mirroring partnership and confirm no further 13189 errors appear in the Windows Event Viewer under Applications and Services Logs > MSSQLSERVER.
  • If the error persists, generate a brand-new self-signed certificate on both instances with CREATE CERTIFICATE using explicit EXPIRY_DATE and matching encryption algorithm settings, then re-establish the endpoint authentication from scratch.