The HotStandby component requires that the server state is switched, when necessary, either automatically or manually by a user.
In production use, the server state is chosen by using automatic state switching, that is, by performing failovers. The automatic state switching can be the responsibility of, for example, the solidDB® High Availability Controller (HAC).
▪Switchover means reversing the roles of the Primary and Secondary when they are running. Switchover can be used for various maintenance purposes.
▪Failover is the action by Secondary for taking up the role of the Primary when the Primary fails.
Performing switchovers
The roles of the servers can be reversed by issuing the following command at the Secondary:
ADMIN COMMAND 'hotstandby switch primary';
or, at the Primary:
ADMIN COMMAND 'hotstandby switch secondary';
The switch commands can be used regardless of whether the two servers are connected. If the servers are connected, the states are reversed; the old Secondary becomes the new Primary, and the old Primary becomes the new Secondary. If the servers are not connected, the old Secondary becomes the new Primary, and the state of the other server is unchanged.
The diagram below shows what happens if you issue the command hsb switch secondary or hsb switch primary when the servers are connected. The command hsb switch primary can be issued only on a server that is in a SECONDARY state (for example, SECONDARY ACTIVE), while the command hsb switch secondary can only be used on a server that is in a PRIMARY state (for example, PRIMARY ACTIVE).
When you issue the command hotstandby switch primary to switch the Secondary server (Srvr2) to Primary, if the Secondary server (Srvr2) is not connected to the other server (Srvr1), an error is returned.
If the two servers are connected, they switch states. In other words, the old Primary (Srvr1) becomes the new Secondary and old Secondary (Srvr2) becomes the new Primary.
If the old Secondary (Srvr2) cannot connect to the other server (Srvr1), both servers switch to SECONDARY ALONE. Even if the AutoPrimaryAlone configuration parameter is set to yes, the new Primary switches to SECONDARY ALONE, not PRIMARY ALONE.
Switching Secondary to Primary
You can switch a Secondary server to PRIMARY state by issuing the command:
ADMIN COMMAND 'hotstandby switch primary';
When the hotstandby switch primary command is issued, it starts a process to switch the state. If the switch process started successfully, the following message is displayed: Started the process of switching the roe to primary. You can monitor the status of the switch by using the command hotstandby status switch.
If you issue a commit after the hotstandby switchprimary command, the commit fails with the error replicated transaction is aborted.
All transactions are terminated during the switch. However, administrative commands (ADMIN COMMAND) are not transactional commands and cannot be rolled back.
Note Administrative commands force the start of a new transaction if one is not already open. To avoid leaving an open transaction, or having a transaction start time be different from what you expected, issue COMMIT WORK after administrative commands.
If a configuration error causes both servers to have the state of PRIMARY (for example, both are PRIMARY ALONE), you can use the command hotstandby switch secondary to switch one of the servers back to a SECONDARY state.
▪If the servers have the same data, normal operations on both servers are resumed.
▪If the servers do not have the same data, the Primary server rejects the connect operation from the Secondary and issues the following message:
14525: HotStandby databases are not properly synchronized.
HotStandby replication is not started. In this case, a full copy of the Primary database is required at the Secondary server. First you must decide which database is correct.
Also, if the 14525 error occurs, the database states do not change. Both servers remain in the same state they were in before the command was issued.
You can switch a Primary server to SECONDARY state by issuing the command
ADMIN COMMAND 'hotstandby switch secondary'
The ADMIN COMMAND 'hotstandby switch secondary' can be useful if two servers have switched states and you want to switch them back to their original states. For example, when the new Secondary comes back in service, you can switch its state back to Primary and switch the new Primary back to Secondary.
When the hotstandby switch secondary command is issued, it starts a process to switch the state. If the switch process started successfully, the following message is displayed: Started the process of switching the role to secondary. You can monitor the status of the switch by using the command hotstandby status switch.
When you execute the hotstandby switchsecondarycommand, if the servers are not already connected to each other, the old Primary tries to connect to the old Secondary.
If the two servers are connected, they switch states. The old Primary becomes the new Secondary and the old Secondary becomes the new Primary.
A failover is performed by executing the command ADMIN COMMAND 'hotstandby set primary alone' at the Secondary.
After all the pending transactions received from the Primary have been processed, server gains the new state once. No transactions are lost, and the database state reflects the state at the Primary just before the failure. However, if the safeness level used is 1-safe, some transactions can be lost in failover.
Running the new Primary in PRIMARY ALONE state
Although the connection to the Secondary server is broken when the Primary server is in the PRIMARY ALONE state, you can run a Primary server with continuous updates to the transaction log. After the Secondary server comes back up, the server in PRIMARY ALONE state can resume sending transactions to the Secondary server.
Procedure
There are three ways to set a server to PRIMARY ALONE state:
▪Issue the following command:
ADMIN COMMAND 'hotstandby set primary alone';
▪Perform a controlled disconnect at either the Primary or the Secondary with the following command:
ADMIN COMMAND 'hotstandby disconnect';
If you do a controlled shutdown by executing ADMIN COMMAND 'shutdown'; on the Secondary, the Secondary disconnects implicitly before shutting down, and the Primary will switch safely to the PRIMARY ALONE state.
▪Set the HotStandby.AutoPrimaryAlone parameter to yes to default to the PRIMARY ALONE state.
When HotStandby.AutoPrimaryAlone=yes, the server is automatically put in PRIMARY ALONE state when the connection to the Secondary is broken. Otherwise, after a server fails, the state of the server remains PRIMARY UNCERTAIN unless the command ADMIN COMMAND 'hotstandby set primary alone' is issued by the HAC, the administrator, or the watchdog program. By default, the HotStandby.AutoPrimaryAlone parameter is set to no, which specifies that the Primary server operating in its PRIMARY ACTIVE state is switched to PRIMARY UNCERTAIN automatically if the Secondary server fails.
Results
▪The PRIMARY ALONE state persists until one of the following occurs:
▪A connection is successfully made to the Secondary server.
▪The server runs out of space for the transaction log.
▪The log size limit (MaxLogSize) is reached.
▪Another command switches the server to another state, such as STANDALONE.
▪The Primary server is shut down.
Important: Do not shut down the Primary simultaneously with commanding Secondary to the PRIMARY ALONE state. The two operations are conflicting and might result in the Secondary gaining the SECONDARY ALONE state instead. These two actions do not coincide in normal operation.
Also, even during the test phase, do not try to simulate Primary failure using shutdown because shutdown is no substitute for failure. Shutdown is a complex distributed operation that involves both Primary and Secondary. Also, after a shutdown the Primary server that starts as a new Secondary cannot catchup with the new Primary. To shut down Primary, the correct sequence is:
1 Perform the switchover.
2 Shut down the new Secondary.
3 The new Primary switches automatically to the PRIMARY ALONE state.
Bringing the secondary server back online
Procedure
To bring the Secondary server back online, connect the Primary with the Secondary server. For more information, see Connecting HotStandby servers.
After you bring a Secondary node online, it might require catchup. Changes in the Primary have accumulated over time. While the Primary was set to PRIMARY ALONE state, the Primary wrote transactions and data to the transaction log. When the Secondary is connected again to the Primary, the pending changes at the Primary are written from the transaction log to the Secondary server for synchronization. While the changes are written to the Secondary, the Secondary is in SECONDARY ALONE state and the Primary is in PRIMARY ALONE state. If you issue the command ADMIN COMMAND 'hsb status connect', the return message indicates whether the servers are performing catchup.
Note If the Primary server was set to the STANDALONE state by using the command hotstandby set standalone, the full database must be copied from the Primary to the Secondary before the Secondary can be put in SECONDARY ACTIVE state. For more information, see Synchronizing primary and secondary servers.
Results
After the Secondary finished processing the pending changes, the Primary and Secondary server states are automatically changed to PRIMARY ACTIVE and SECONDARY ACTIVE.
