How to fix Error 9691 on Windows Server 2025 / Windows 11

by

Troubleshooting Guide: SQL Server Error 9691 – Endpoint Stopped Listening

Why This Error Happens

Error 9691 occurs when a SQL Server endpoint (such as the Database Mirroring, Service Broker, or TCP endpoint referenced by %S_MSG) encounters an unrecoverable network-level failure and is forced to shut down its listener. This is typically caused by the underlying TCP port being seized by another process, a bound network interface becoming unavailable, exhaustion of system socket resources, or corruption in the SQL Server network configuration. It can also occur if the service account lacks sufficient permissions to bind to the specified port or IP address after a Windows Server 2025 security policy update.

Step-by-Step Resolution Guide

Step 1: Identify the Failed Endpoint and Port

  • Check the SQL Server Error Log for the full message text to identify which endpoint triggered the error (e.g., Mirroring, Service Broker, or a custom TCP endpoint).
  • Run the following query to list all endpoints and their current state:
  SELECT name, protocol_desc, type_desc, state_desc, port 
  FROM sys.tcp_endpoints;
  • Note the port number associated with the failed endpoint for the next step.

Step 2: Check for Port Conflicts

  • Open Command Prompt as Administrator and run:
  netstat -ano | findstr <PORT_NUMBER>
  • Identify the PID using the port and cross-reference it in Task Manager (Details tab) to see if a non-SQL process has claimed it.
  • If a conflict exists, either stop the conflicting process or reassign the SQL Server endpoint to a new port using:
  ALTER ENDPOINT [EndpointName] AS TCP (LISTENER_PORT = 5099);

Step 3: Verify Service Account Permissions

  • Open Local Security Policy (secpol.msc) on Windows Server 2025 / Windows 11.
  • Navigate to Security Settings > Local Policies > User Rights Assignment.
  • Confirm the SQL Server service account has the “Access this computer from the network” right and is not restricted by the “Deny access to this computer from the network” policy.
  • Restart the SQL Server service via services.msc to reload permission changes.

Step 4: Restart the Endpoint and Validate Firewall Rules

  • Manually restart the endpoint using T-SQL:
  ALTER ENDPOINT [EndpointName] STATE = STARTED;
  • Open Windows Defender Firewall with Advanced Security and ensure an inbound rule exists allowing traffic on the SQL Server port (default 1433, or custom mirroring/broker ports like 5022/4022).
  • Add a new rule if missing:
  netsh advfirewall firewall add rule name="SQLServerEndpoint" dir=in action=allow protocol=TCP localport=5022

Step 5: Check for Corrupted Network Configuration

  • If the issue persists, open SQL Server Configuration Manager.
  • Navigate to SQL Server Network Configuration > Protocols for [Instance] and verify TCP/IP is enabled with correct IP binding (check the IPAll section for the correct TCP Port value).
  • Restart the SQL Server service to apply changes and clear any cached network binding corruption.

Alternative or Verification Steps

  • To verify the fix, re-run SELECT name, state_desc FROM sys.tcp_endpoints; and confirm the state shows STARTED rather than STOPPED.
  • If the endpoint repeatedly fails to start, as a workaround, drop and recreate the endpoint entirely using CREATE ENDPOINT with explicit port and IP bindings to clear any residual corrupted metadata.