...
Federated architecture best fits customers that execute high load (thousands of transactions per second or more) in their gateways.
The cell environment implements the federated architecture by distributing DPOD's Store and DPOD's agents across different federated servers.
...
- Cell Manager - a DPOD server (usually virtual) that manages all Federated Cell Members (FCMs), as well as providing central DPOD services such as the Web Console, reports, alerts, resource monitoring, etc.
- Federated Cell Members (FCMs) - DPOD servers (usually physical with very fast local storage) that include Store data nodes and agents (Syslog and WS-M) for collecting, parsing and storing data.
The cell environment does not replicate any data between the members, so adding more members will not provide any HA / DR capabilities.
The following diagram describes the cell environment:
...
- Before installing a cell environment, make sure to complete the sizing process with IBM Support Team to get recommendations for the hardware and architecture suitable for your requirements.
- DPOD cell manager and federated cell members must be of the same version (minimum version is 1.0.8.6).
- DPOD cell manager is usually virtual and can be installed in both Appliance Mode or Non-Appliance Mode with Medium Load architecture type, as detailed in the Hardware and Software Requirements.
- DPOD federated cell members (FCMs) can be one of the following:
- Physical servers installed in Non-appliance Mode (based on RHEL) with High_20dv architecture type, as detailed in the Hardware and Software Requirements.
Physical servers are used when the cell is required to process high transactions per second (TPS) load. - Virtual servers installed in Non-appliance Mode with Medium architecture type or higher, as detailed in the Hardware and Software Requirements.
Virtual servers are used when the cell is required to process moderate transactions per second (TPS) load, or when the cell is part of a non-production environment where the production cell uses physical servers (to keep environments architecture similar).
- Physical servers installed in Non-appliance Mode (based on RHEL) with High_20dv architecture type, as detailed in the Hardware and Software Requirements.
- All DPOD cell members must be identical - only physical or only virtual (cannot mix physical and virtual cell members in the same cell), and with the same resources (CPUs, RAM, disk type and storage capacity).
- Physical federated cell members with 4 CPU sockets and NVMe disks require special disks and mount points configuration to ensure performance. See Configuring Cell Members with 4 CPU Sockets and NVMe Disks.
- Each cell component (manager / FCM) should have two network interfaces:
- External Internal network interface - dedicated for DPOD users to access the Web Console (on the cell manager) and for inter-communication between the cell components.
- External network interface - for communicating with the rest of the network. This includes users accessing the DPOD Web Console (on the cell manager), communication between DPOD and the Monitored Gateways (on both the cell manager and the members).
- Internal network interface - for internal DPOD components inter-communication (should be a 10Gb Ethernet interface).
- This design , communication with DNS, NTP, SMTP, LDAP, and anything else on the network.
- This design was driven by customer requirements and allows separation between the two types of communications, which may be used to enhance the security (e.g.: deny end-users from being able to access the inter-cell communication).
- We recommend having 2 different VLANs with different subnets as this makes it easier to configure the servers without using static routing and that all the internal network interfaces have IP addresses which belong to a single subnet (the internal subnet), and also all the external network interfaces have IP addresses which belong to a single subnet (the external subnet). Having an internal subnet that is different from the external subnet makes it easier to configure the servers without using static routing and easier to configure the network firewall rules.
- Network ports should be opened in the network firewall as detailed below:
...
From
...
To
...
Ports (Defaults)
...
Protocol
...
Usage
...
DPOD Cell Manager (external IP address)
...
Each Monitored Device
...
5550 (TCP)
...
HTTP/S
...
Monitored device administration management interface.
If the SOMA port is different than 5550 - the port should be changed accordingly.
...
DPOD Cell Manager (external IP address)
...
TCP and UDP 53
...
DNS services. Static IP address may be used.
...
DPOD Cell Manager (external IP address)
...
NTP Server
...
123 (UDP)
...
NTP
...
Time synchronization
...
DPOD Cell Manager (external IP address)
...
Organizational mail server
...
25 (TCP)
...
SMTP
...
Send reports by email
...
DPOD Cell Manager (external IP address)
...
LDAP
...
TCP 389 / 636 (SSL).
TCP 3268 / 3269 (SSL)
...
LDAP
...
Authentication & authorization. Can be over SSL.
...
NTP Server
...
DPOD Cell Manager (external IP address)
...
123 (UDP)
...
NTP
...
Time synchronization
...
Users IPs
...
DPOD Cell Manager (external IP address)
...
443 (TCP)
...
HTTP/S
...
DPOD's Web Console
...
Admins IPs
...
DPOD Cell Manager (external IP address)
...
22 (TCP)
...
TCP
...
SSH
...
Each DPOD Federated Cell Member (external IP address)
...
TCP and UDP 53
...
DNS services
...
Each DPOD Federated Cell Member (external IP address)
...
NTP Server
...
123 (UDP)
...
NTP
...
Time synchronization
...
Each Monitored Device
...
Each DPOD Federated Cell Member (external IP address)
...
60000-60003 (TCP)
...
TCP
...
SYSLOG Data
...
Each Monitored Device
...
Each DPOD Federated Cell Member (external IP address)
...
60020-60023 (TCP)
...
HTTP/S
...
WS-M Payloads
...
NTP Server
...
Each DPOD Federated Cell Member (external IP address)
...
123 (UDP)
...
NTP
...
Time synchronization
...
Admins IPs
...
Each DPOD Federated Cell Member (external IP address)
...
22 (TCP)
...
TCP
...
SSH
Cell Manager Installation
Prerequisites
- Make sure to meet the prerequisites listed at the top of this page.
- For Non-appliance Mode, follow the procedure: Prepare Pre-Installed Operating System.
DPOD Installation
- For Appliance Mode, follow the procedure: Appliance Installation.
For Non-appliance Mode, follow the procedure: Non-Appliance Installation. - During installation, when prompted to choose the data disk type (SSD / non SSD), choose the cell members disk type (should be SSD) instead of the cell manager disk type.
- During installation, when prompted to choose the IP address for the Web Console, choose the IP address of the external network interface.
- Install the following software package (RPM): bc
- Execute the following operating system performance optimization commands and reboot the server:
| Code Block | ||||
|---|---|---|---|---|
| ||||
sed -i 's/^NODE_HEAP_SIZE=.*/NODE_HEAP_SIZE="2G"/g' /etc/init.d/MonTier-es-raw-trans-Node-1
/app/scripts/tune-os-parameters.sh
reboot |
Federated Cell Member Installation
The following section describes the installation process of a single Federated Cell Member (FCM). Please repeat the procedure for every FCM installation.
Prerequisites
...
Most Linux-based OS use a local firewall service (e.g.: iptables / firewalld). Since the OS of the Non-Appliance Mode DPOD installation is provided by the user, it is under the user's responsibility to allow needed connectivity to and from the server.
Configure the local firewall service to allow connectivity as described in the prerequisites section at the top of this page.
...
- A diagram demonstrating this is available in Firewall Rules for DPOD Cell Environment.
- Network rules should be defined as detailed in Firewall Rules for DPOD Cell Environment.
Cell Manager Installation
Prerequisites
- Make sure to meet the prerequisites listed at the top of this page.
- For Non-appliance Mode, follow the procedure: Prepare Pre-Installed Operating System.
DPOD Installation
- For Appliance Mode, follow the procedure: Appliance Installation.
For Non-appliance Mode, follow the procedure: Non-Appliance Installation.
During installation, when prompted to choose the data disk type (SSD / non SSD), choose the cell members disk type (should be SSD) instead of the cell manager disk type.
During installation, when prompted to choose the IP address for the Web Console, choose the IP address of the external network interface. - Once installation is done, execute the following operating system performance optimization commands and reboot the server:
| Code Block | ||||
|---|---|---|---|---|
| ||||
sed -i 's/^NODE_HEAP_SIZE=.*/NODE_HEAP_SIZE="2G"/g' /etc/init.d/MonTier-es-raw-trans-Node-1
/app/scripts/tune-os-parameters.sh
reboot |
Federated Cell Member Installation
The following section describes the installation process of a single Federated Cell Member (FCM). Please repeat the procedure for every FCM installation.
Prerequisites
- Make sure to meet the prerequisites listed at the top of this page.
- Follow the procedure: Prepare Pre-Installed Operating System.
- The cell member server should contain disks according to the recommendations made in the sizing process with IBM Support Team, which includes disks for OS, install, and data (one for /data and 6 to 9 additional disks for /data2/3/4...).
- Physical federated cell members with 4 CPU sockets and NVMe disks require special disks and mount points configuration to ensure performance. See Configuring Cell Members with 4 CPU Sockets and NVMe Disks.
DPOD Installation
- Physical servers should use RHEL as the operating system (and not CentOS).
- Use Non-appliance Mode and follow the procedure: Non-Appliance Installation
The During installation, the four-letter Installation Environment Name should be identical to the one that was chosen during the Cell Manager installation.
During installation, when prompted to choose the IP address for the Web Console, choose the IP address of the external network interface. - Install the following software package (RPM): numactl pciutils nvme-cli
- Execute .
- Once installation is done, execute the following operating system performance optimization commands command and reboot the server:
| Code Block | ||||
|---|---|---|---|---|
| ||||
/app/scripts/tune-os-parameters.sh reboot |
...
- In case of physical federated cell members with 4 CPU sockets and NVMe disks - use the information gathered at Configuring Cell Members with 4 CPU Sockets and NVMe Disks to map the mount point with the proper disk:
| Mount Points | Disks |
|---|---|
| /data2, /data22 and /data222 (if exists) | Disks connected to NUMA node 1 |
| /data3, /data33 and /data333 (if exists) | Disks connected to NUMA node 2 |
| /data4, /data44 and /data444 (if exists) | Disks connected to NUMA node 3 |
- For all other types of federated cell members servers - you may map the mount points to any disk.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
sed -i 's/^NODE_HEAP_SIZE=.*/NODE_HEAP_SIZE="64G"/g' /etc/init.d/MonTier-es-raw-trans-Node-2 sed -i 's/^NODE_HEAP_SIZE=.*/NODE_HEAP_SIZE="64G"/g' /etc/init.d/MonTier-es-raw-trans-Node-3 sed -i 's/^NODE_HEAP_SIZE=.*/NODE_HEAP_SIZE="64G"/g' /etc/init.d/MonTier-es-raw-trans-Node-4 |
Restarting Services of the Federated Cell Member
Stop all the application services using app-util.sh → option 2 (stop all)
Start all the application services using app-util.sh → option 1 (start all)
Cell Member Federation Verification
After a successful federation, you will be able to see the new federated cell member in the Manage → System → Nodes page. For example:
...
Configure the monitored gateways to use the federated cells agents. Please follow instructions on Adding Monitored DevicesGateways.