High Availability Guide : Watchdog sample : Failure situations and Watchdog actions : Communication link between Primary and Secondary is down
  
Communication link between Primary and Secondary is down
Scenario
The connection between the Primary and Secondary server is broken.
The Primary will switch itself to PRIMARY UNCERTAIN state. (If AutoPrimaryAlone is set to Yes, then the server will switch itself to PRIMARY ALONE state.)
Note If the Primary server sends a commit message to the Secondary and then detects the failure of the Secondary, the Primary server relies on the Watchdog or the administrator to indicate how the Primary server is to proceed. This is because the Primary server is unable to detect whether the transaction was committed or rolled back in the Secondary before the Secondary server failed.
Until the Primary server receives a command from the Watchdog or the administrator, it no longer accepts transactions. At this stage, in order for the Primary server to continue operations, the Watchdog or administrator can set the Primary server to PRIMARY ALONE state.
Remedy
The Primary server can continue operations even when its link to the Secondary server is down. If the Primary is not already in PRIMARY ALONE state, then switch the Primary to the PRIMARY ALONE state. Once the link between the Primary and Secondary is restored, synchronize the databases.
1 Watchdog instructs Server 1:
HSB SET PRIMARY ALONE
HSB SET STANDALONE
2 After the connection between primary and secondary is fixed, Watchdog instructs Server1:
HSB NETCOPY
HSB CONNECT
3 If the transaction log fills up, you may have to switch the PRIMARY ALONE server to STANDALONE. In that case, you will also need to execute HSB NETCOPY before you reconnect the servers. If the transaction log does not fill up, then you must skip the NETCOPY command.
Symptoms
The Primary server has no Secondary connected and the state is PRIMARY UNCERTAIN or PRIMARY ALONE.
How to recover when the communication link between the Primary and Secondary is down
About this task
To recover from the scenario where the connection between the Primary and Secondary server is broken:
Procedure
1 Fix the network connection between the Primary and Secondary servers.
2 Check the state of the Primary server using the command:
ADMIN COMMAND 'hotstandby state';
3 If the state of the Primary server is PRIMARY ALONE, reconnect the Primary to the Secondary using the command:
ADMIN COMMAND 'hotstandby connect';
4 If the state of the Primary server is STANDALONE, then:
a Copy the database from the Primary to the Secondary. Read Synchronizing primary and secondary servers for details.
b Before using the command ADMIN COMMAND 'hotstandby netcopy'; be sure that the Secondary is up and running and is ready to receive the netcopy. Also, make sure that you set the Primary server's state to PRIMARY ALONE.
c Reconnect the Primary to the Secondary using the command:
ADMIN COMMAND 'hotstandby connect';
Further scenarios when the communication link between the Primary and Secondary is down
If an application receives error message 10047 or 14537 from the Primary:
Try to connect to the Secondary to check if it is switched as the new Primary.
If the old Secondary is not switched as the new Primary, see scenario in Primary is down.
See also
Failure situations and Watchdog actions