How to fix Error 13186 on Windows Server 2025 / Windows 11

by

Why This Error Happens

This error occurs when SQL Server attempts to use a certificate for encrypting the Database Mirroring or Always On Availability Group endpoint, but the RSA private key length exceeds the maximum size supported by the Windows Cryptographic API (CAPI) handshake used internally by SQL Server’s transport layer. SQL Server 2025 enforces stricter validation on key exchange, meaning legacy certificates generated with 4096-bit or larger keys (or keys created using deprecated CSP providers instead of CNG/KSP) will fail the handshake. This is commonly triggered after migrating certificates from older SQL Server instances or importing a certificate issued by a corporate CA with non-standard key lengths.

Step-by-Step Resolution Guide

Step 1: Verify the Current Certificate’s Key Size

  • Open SQL Server Management Studio (SSMS) and run the following query to inspect the certificate used by the endpoint:
SELECT name, subject, pvt_key_encryption_type_desc, 
       key_length 
FROM sys.certificates 
WHERE name = 'YourEndpointCertName';
  • Confirm whether the key_length value exceeds 3072 bits, as SQL Server 2025 endpoints only support up to 3072-bit RSA keys for mirroring/AG encryption.

Step 2: Generate a Compliant Certificate

  • Use makecert or preferably PowerShell with the CNG provider to generate a new certificate with a supported key size:
New-SelfSignedCertificate -DnsName "SQLServerNode1" -CertStoreLocation "cert:\LocalMachine\My" -KeyLength 2048 -KeyAlgorithm RSA -Provider "Microsoft Software Key Storage Provider"
  • Export the certificate (.pfx) with the private key, ensuring it is placed in C:\Certs\ for import on all partner nodes (primary, mirror, witness, or AG replicas).

Step 3: Reassign the Certificate to the Endpoint

  • In SSMS, drop the old certificate binding and recreate it:
ALTER ENDPOINT [Mirroring]
STATE = STOPPED;

CREATE CERTIFICATE NewMirrorCert
FROM FILE = 'C:\Certs\NewMirrorCert.cer'
WITH PRIVATE KEY (FILE = 'C:\Certs\NewMirrorCert.pvk', DECRYPTION BY PASSWORD = 'YourStrongPassword');

ALTER ENDPOINT [Mirroring]
STATE = STARTED
AS TCP (LISTENER_PORT = 5022)
FOR DATABASE_MIRRORING (AUTHENTICATION = CERTIFICATE NewMirrorCert, ENCRYPTION = REQUIRED ALGORITHM AES);
  • Repeat the certificate import and endpoint binding on all partner/replica servers to ensure matching public keys.

Step 4: Restart SQL Server Services

  • Restart the SQL Server (MSSQLSERVER) service via services.msc or run:
Restart-Service -Name MSSQLSERVER -Force
  • This ensures the endpoint reloads the new certificate context and clears any cached key exchange handshake data.

Alternative or Verification Steps

  • Check the SQL Server Error Log (ERRORLOG in C:\Program Files\Microsoft SQL Server\MSSQL17.MSSQLSERVER\MSSQL\Log\) for successful endpoint startup messages with no further 13186 entries.
  • If the issue persists, bypass custom CA certificates entirely by using a self-signed certificate generated directly via T-SQL (CREATE CERTIFICATE ... WITH SUBJECT = 'Mirroring Cert'), which defaults to a SQL Server-compliant key size and provider.