High Availability Guide : Failure handling with High Availability Controller (HAC) : Server is unresponsive to external clients
  
Server is unresponsive to external clients
Scenario
Connection to the server fails or hangs forever. Servers are in PRIMARY ACTIVE, and SECONDARY ACTIVE state, but clients cannot connect to them and they cannot execute transactions.
Recovery
Regardless of the state of unresponsive server, HAC executes a script as configured in the solidhac.ini configuration file with the LocalDB.UnresponsiveActionScript parameter. The script is started with single parameter including the process id (pid) of unresponsive solidDB® process.
Typical solution is to terminate the process identified by the process id. In such a case, the recovery proceeds automatically as follows:
If the unresponsive server was in Primary role:
1 Upon the connection failure, the Secondary database (node 2) moves automatically to the SECONDARY ALONE state.
2 The HAC instance on the Secondary database (node 2) notices that the Primary database process (node 1) has terminated and sets the Secondary database (node 2) to the PRIMARY ALONE state.
3 In parallel with the above task, the HAC instance on the Primary database (node 1) restarts the old Primary database, which enters the SECONDARY ALONE state.
4 The HAC instance on the Secondary database (node 2) initiates the process of connecting the Primary and Secondary database.
5 A catchup is made.
Optionally, the connection process includes a netcopy operation from the Secondary database (node 2) to the Primary database (node 1).
If the unresponsive server was in Secondary role:
1 Upon the connection failure, the Primary database (node 1) moves automatically to the PRIMARY UNCERTAIN state, or if the AutoPrimaryAlone parameter is enabled, to the PRIMARY ALONE state.
2 The HAC instance on the Primary database (node 1) notices that the Secondary database process (node 2) has terminated.
3 If the Primary database was set to the PRIMARY UNCERTAIN state in step 1, HAC sets it now to the PRIMARY ALONE state.
4 In parallel with the above task, the HAC instance on the Secondary database (node 2) restarts the old Secondary database, which enters the SECONDARY ALONE state.
5 The HAC instance on the Primary database (node 1) initiates the process of connecting the Primary and Secondary database.
6 A catchup is made.
Optionally, the connection process includes a netcopy operation from the Primary database to the Secondary database.
See also
Failure handling with High Availability Controller (HAC)