solidDB Help : High availability : solidDB HotStandby : High Availability Controller : Failure handling with High Availability Controller (HAC) : Summary of failure scenarios and HAC actions
  
Summary of failure scenarios and HAC actions
The following table summarizes possible failure scenarios and the corresponding actions that are taken by the High Availability Controller (HAC).
HAC actions are recorded in the HAC message log file (hacmsg.out). The table includes the key phrase that is written to the log when a HAC action is implemented.
 
Scenario
HAC action
Intended result
HAC log message
Both servers are in a SECONDARY state.
Executes command on local server that is similar to:
ADMIN COMMAND 'hsb set primary alone'
Server with the more up-to-date database is set to PRIMARY ALONE state.
CHOOSE_SET_PRIMARY
One of the following situations occurs:
Local server has failed
Local server is not started and HAC is configured to start server
HAController.EnableDBProcessControl is enabled in solidhac.ini file, and user runs the following command: ADMIN COMMAND 'hac restartdb'
Executes the script that is specified in LocalDB.StartScript in solidhac.ini.
Depends on content of script.
START_SERVER
HAController.EnableDBProcessControl is enabled in solidhac.ini file, and user runs the following command on the local server: ADMIN COMMAND 'hac shutdowndb'
Executes command on local server that is similar to:
ADMIN COMMAND 'shutdown force'
Local server is shut down.
SHUTDOWN_LOCAL
One of the following situations occurs:
Remote server has failed, is not available, or is in backup server mode, and local server is in a SECONDARY state.
Remote server has failed and local server is in PRIMARY UNCERTAIN state.
Remote server is in backup server mode and local server is in PRIMARY UNCERTAIN state.
Remote server is in backup server mode and local server is in STANDALONE state.
Remote server is in a SECONDARY state and local server is in STANDALONE state.
HotStandby connection between servers has failed and, either ERE is not configured (default), or ERE is configured and can be connected, see External Reference Entity (ERE).
Executes command on local server that is similar to:
ADMIN COMMAND 'hsb set primary alone'
Local server is set to PRIMARY ALONE state.
SET_LOCAL_PRIMARY_ALONE
HotStandby connection between servers has failed, ERE is configured but cannot be connected, see External Reference Entity (ERE).
Executes a command on local server that is similar to:
ADMIN COMMAND 'hsb set secondary alone'
Local server is set to SECONDARY ALONE state.
SET_SECONDARY_ALONE
Servers are ready to connect (local server is in a PRIMARY state and remote server is in a SECONDARY state).
Executes command on local server that is similar to:
ADMIN COMMAND 'hsb connect'
The servers are connected and HotStandby replication is started.
DO_HSB_CONNECT
One of the following situations occurs:
Remote server is in backup server mode and local server is in a PRIMARY state
Establishing HotStandby connection failed, local server is in a PRIMARY state and remote server is in a SECONDARY state.
Executes commands on local server that is similar to:
ADMIN COMMAND 'hsb set primary alone'
(to ensure that local server is in a PRIMARY state)
ADMIN COMMAND 'hsb netcopy'
(copies whole database from local server to remote server)
ADMIN COMMAND 'hsb status copy'
(monitors the progress of the database copy until the copy is complete or fails)
The databases on the local and remote servers are synchronized.
DO_NETCOPY
User runs the following command on local server that is in a SECONDARY state:
ADMIN COMMAND 'hac switchhsb'
Executes command on local server that is similar to:
ADMIN COMMAND 'hsb switch primary'
The local server is set to a PRIMARY state.
SWITCH_TO_PRIMARY
User runs the following command on local server that is in a PRIMARY state:
ADMIN COMMAND 'hac switchhsb'
Executes command on local server that is similar to:
ADMIN COMMAND 'hsb switch secondary'
The local server is set to a SECONDARY state.
SWITCH_TO_SECONDARY
Both servers in a cluster are in a PRIMARY state where they both would accept updates to the database, thus endangering the database consistency.
Executes series of commands:
1 Close remote server and switch the state to SECONDARY ALONE.
2 Close local server and switch the state to SECONDARY ALONE.
3 Open remote server for client connections.
4 Open local server for client connections.
Cluster is reset so that the server with the most up-to-date database is set to PRIMARY ACTIVE state, and the other server is set to SECONDARY ACTIVE state, and replication between the server is restarted.
RESET_SERVICE
Local HotStandby server process is alive but the server doesn't behave correctly, (accept client connections, or execute transactions, for example).
The action is enabled only if LocalDB.EnableApplicationConnTest and LocalDB.EnableUnresponsiveActions are enabled, and LocalDB.UnresponsiveActionScript is set.
Executes the script that is specified in LocalDB.UnresponsiveActionScript in solidhac.ini.
Depends on the content of script
UNRESPONSIVE_PROCESS_ACTION
Go up to
Failure handling with High Availability Controller (HAC)