8. Configuring an LVS Cluster

You set up and maintain an LVS cluster from the LVS routers, by editing the configuration file and starting or restarting the pulse daemon. Specifically, the steps are:

  1. On the primary router, edit the configuration file /etc/lvs.cf.

  2. Copy the edited configuration file to the backup router.

  3. Start (or restart) the pulse daemon first on the primary router, then on the backup router.



You can perform these steps from the shell, by editing the configuration file with the editor of your choice. The shell commands for starting, restarting, and stopping the pulse daemon are:

    /etc/rc.d/init.d/pulse start
    /etc/rc.d/init.d/pulse restart
    /etc/rc.d/init.d/pulse stop

The pulse daemon starts or restarts the other LVS cluster daemons as needed, which obtain their configuration information, directly or indirectly, from the current configuration file.

If you stop pulse (in order to shut down the cluster), stop it on the backup router first. This will prevent the backup router from possibly taking over the active role.

Alternatively, you can use piranha to set up, monitor, and administer your LVS cluster. The entry fields on its windows set or change lines in /etc/lvs.cf, and there are buttons to start, stop, and restart the cluster. Use of piranha requires that the X Window System be configured on the LVS routers. An advantage of using piranha is that it synchronizes the configuration file on the primary and backup routers, and you perform all administrative tasks from the active router.

The next section describes the LVS cluster configuration file. Read this section if you want to edit this file manually. If you chose to use piranha, read Section 8.2.

8.1. Editing the Configuration File

The /etc/lvs.cf file has three sections. The global section, described in Table 3, sets up the LVS routers and specifies networking and heartbeat parameters. There is one set of parameters for the cluster. The per-virtual-server section, described in Table 4, defines virtual server addresses, sets up the associations between virtual servers and real servers, and specifies job-scheduling parameters. There is a separate set of parameters for each defined virtual server. The per-real-server section, described in Table 5, defines the real servers routed to from each virtual server. There is one set of these parameters for each virtual server.

Table 3. Setting Global Parameters

Parameter Description
primary = Enter the IP address of the adapter connecting the primary LVS router to the public network.
backup = Enter the IP address of the adapter connecting the backup backup LVS router to the public network.
heartbeat_port = Enter the port number used for the heartbeat on the primary and backup LVS routers.
keepalive = Enter the number of seconds between heartbeats.
deadtime = Enter the number of seconds to wait before declaring a non-responding router dead and initiating failover.
rsh_command = [rsh|ssh] Enter the command family to use for synchronizing the configuration files on the primary and backup routers. Important: as described in Table 2, you must enable the selected command on the primary and backup routers.
network = [nat|direct|tunnel] Currently, only Network Address Translation (nat) is supported.
nat_router = Enter the floating IP address and device of the NAT router. This IP address must be the default route used by each real server to communicate with the active LVS router. The IP address is aliased to the device (for example, eth1:1) connecting the LVS routers to the private network of real servers. The device must be the same (i.e., eth1) on both LVS routers.

Table 4. Setting Per-Virtual-Server Parameters

Parameter Description
name Enter a unique identifier for the virtual server.
address = Enter the virtual server's IP address: a floating IP address that has been associated with a fully-qualified domain name.
active = [0|1] Enable (1) or disable (0) this address.
load_monitor = [uptime|ruptime|rup] Select the tool (default uptime) that will be used by the active router to monitor the workload of real servers. Important: as described in Table 2, unless you enable the selected command on the real servers, the scheduling algorithms that use dynamic load information will apply the assigned weight statically rather than adjust the weight from load information. If you select the default (uptime), the tool you specified for "rsh_command" is used to log in to the real servers. This tool must be enabled on the real servers.
timeout = Enter the number of seconds (default 10) that must lapse before a real server determined to be dead is removed from the routing table.
reentry = Enter the number of seconds (default 180) that a restored real server must remain alive before being re-added to the routing table.
port = [http/80|ftp/21] Enter the listening port on this virtual server: http for 80 (the default) or ftp for 21. Or you can enter the numbers.
scheduler = [wlc|lc|wrr|rr] Select the scheduling algorithm (default wlc) for distributing jobs from this virtual server to the real servers. The choices are described in Table 1.

Table 5. Setting Per-Real-Server Parameters

Parameter Description
name Enter a unique name for the real server.
address = Enter the IP address of the real server on the private network.
active = [0|1] Enable (1) or disable (0) the real server.
weight = Enter an integer (default is 1) specifying this server's processing capacity relative to that of other real servers. For example, a server assigned 2000 has twice the capacity of a server assigned 1000. The weighted scheduling algorithms adjust this number dynamically based on workload.

8.2. Using the Piranha Configuration Tool

To start piranha, become root and type piranha &. When you do, its main window opens. The window has four tabs:

Controls/Monitoring -- The initial tab. Used to start/stop/restart the cluster daemons and monitor the runtime status. See Section 8.2.1.

Global Settings -- Use to set the IP address of the primary LVS router and NAT router. See Section 8.2.2.

Redundancy -- Use to set the address of the backup LVS router and set the heartbeating parameters. See Section 8.2.3.

Virtual Servers -- Use to set service addresses and set up routing between service addresses and real Web/FTP server hosts. See Section 8.2.4.

Piranha windows have one or more of these buttons along the bottom:

OK -- Apply changes and exit from the window.

Apply -- Apply changes without exiting the window.

Close -- Exit from the window without applying changes.

8.2.1. Controls/Monitoring Tab

Field/Button Description
Start/Stop When cluster daemons are not running, this button is labeled Start: click it to start the cluster. When the daemons are running, the button is labeled Stop: click it to stop the cluster daemons.
Add pulse daemon to this runlevel Select to start pulse at system boot at this runlevel.
Update information now Click to display current information in the kernel routing table.
Auto-update Select to display kernel table routing information automatically at the specified interval.

8.2.2. Global Settings Tab

Field/Button Description
Primary LVS server IP Contains the public IP address of the primary LVS router.
NAT Router IP Contains the floating IP address associated with the network adapter connecting the virtual server with the network of real servers. This address is the gateway used by the real servers to communicate with the virtual server. If this node fails, this address migrates to the backup LVS node.
NAT Router Device Names the device with which the NAT router IP address is associated. This is done via address aliasing. For example, the NAT router IP may be aliased to physical device eth1 as eth1:1.
Sync tool Select the tool (rsh or ssh) for synchronizing files on the primary and backup LVS routers. The tool you select must be enabled, such that the LVS routers can log in to one another without needing to enter a password. For general procedures, see Table 2.

8.2.3. Redundancy Tab

Field/Button Description
Enable redundant server Select to enable failover.
Redundant LVS server IP Contains the public IP address of the backup LVS node.
Heartbeat interval (seconds) Contains the number of seconds between heartbeats: the interval at which the backup LVS node checks to see if the primary LVS node is alive.
Assume dead after (seconds) If this number of seconds lapses without a response from the primary LVS node, the backup router initiates failover. During failover, the backup router claims all virtual server IPs being processed by the primary LVS router, and broadcasts gratuitous ARPs advertising its MAC address as the destination of packets addressed to the primary router.
Heartbeat port Enter the port used for the heartbeat on the primary and backup LVS routers.

8.2.4. Virtual Servers

This screen displays a row of information for each currently defined virtual server. Click a row to select it. The buttons on the right side of the screen apply to the currently selected virtual server. Click Delete to remove the selected virtual server.

Field/Button Description
Status Displays Active or Down. Click Disable to take down a selected active virtual server; click Activate to enable a selected down virtual server. After changing the status, you must restart the pulse daemon to make the change take affect. To do this, go to the Controls/Monitoring tab, click the Stop button (its name changes to Start), then click Start.
Name Contains a unique identifier for the virtual server.
Port Contains the number of the listen port for incoming requests for service.
Protocol Currently, only TCP is supported.

8.2.4.1. Add/Edit a Virtual Server

Click the Add button to create a new, undefined virtual server. Click the Edit button (or double-click the row) to define or modify a virtual server. Click the Real Servers tab to view or modify the real servers associated with the selected virtual server.

Field/Button Description
Name Enter a unique identifier for this virtual server.
Application Click to select HTTP or FTP.
Port Enter the number of the listen port for incoming requests for service.
Address Enter the floating IP address where requests for service arrive. This address will have been associated with the fully-qualified domain name to which requests for service are addressed.
Device Name the adapter device connecting the LVS routers to the public network, with which the floating IP address is associciated via IP aliasing (For example, eth0:1). If the active router fails, the virtual server address and port are failed-over to this device on the backup router, which becomes the active router.
Re-entry Time Enter the number of seconds (default is 180) that a failed real server associated with this virtual server must remain alive before it will be re-added to the cluster.
Service timeout Enter the number of seconds (default is 10) within which a real server must respond to a redirected request. If a host takes longer than this, it is declared dead and removed from the cluster.
Every two seconds the nanny process on the active router sends a heartbeat (essentially a ping) to each real server. If this succeeds, then a service connect (essentially a telnet conection) is sent out on the specified service port. If nanny gets back any response from a real server, it and the service are assumed to be alive. If "Service timeout" seconds lapse without a response, then the real server is presumed to be dead and is dropped from the routing table. But nanny continues to monitor the server/service by sending out heartbeats every two seconds. If, after a successful ping, a dropped server remains alive for "Re-entry Time" number of seconds, the server is re-added to the kernel routing table.
Load monitoring tool Select the tool to use (uptime, rup, or ruptime) for determining the workload on the real servers. The general procedures for enabling these tools are described in the table in Table 2.
Scheduling Select the algorithm for routing requests from this virtual server to the real servers. The choices are described in Table 1.

8.2.5. Real Servers

This screen displays a row of information for each currently defined real server. Click a row to select it. The buttons on the right side of the screen apply to the currently selected row. Click Delete to remove the selected real server.

Field/Button Description
Status Displays Active or Down. Click Disable to take down a selected active virtual server; click Activate to enable a selected down virtual server. After changing the status, you must restart the pulse daemon to make the change take affect. To do this, go to the Controls/Monitoring tab, click the Stop button (its name changes to Start), then click Start.
Name Displays the server's name in the cluster.
Address Displays the server's IP address.

8.2.5.1. Add/Edit a Real Server

Click the Add button to create a new, undefined real server. Click the Edit button (or double-click the row) to define or change the selected real server.

Field/Button Description
Name Enter a descriptive name.
Address Enter the IP address of the real server on the private network.
Weight Assign an integer to indicate this host's processing capacity relative to that of other hosts in the pool.