...
Federated architecture best fits customers that execute high load (thousands of transactions per second or more) in their gateways, where the vast majority of the transactions is executed on-premise.
The The cell environment implements the federated architecture by distributing DPOD's Store and DPOD's processing (using DPOD's agents ) across different federated servers.
The cell environment has two main components:
- Cell Manager - a DPOD server (usually virtual or physical) that manages all Federated Cell Members (FCMs), as well as providing central DPOD services such as the Web Console, reports, alerts, resource monitoring, etc.
- Federated Cell Member Members (FCMFCMs) - a DPOD server servers (usually physical with very fast local storage) that includes include Store data nodes and agents (Syslog and WS-M) for collecting, parsing and storing data. There could be one or more federated cell members per cell.
The following diagram describes the Cell Environmentcell environment:
The following procedure describes the process of establishing a DPOD cell environment.
Prerequisites
...
Prerequisites
- Before installing a cell environment, make sure to complete the sizing process with IBM Support Team to get recommendations for the hardware and architecture suitable for your requirements.
- DPOD cell manager and federated cell members must be of the same version (minimum version is is 1.0.8.56).
- DPOD cell manager is usually virtual and can be installed in both Appliance Mode or Non-Appliance Mode with Medium Load architecture both Appliance Mode or Non-Appliance Mode with Medium Load architecture type, as detailed in the the Hardware and Software Requirements. The manager server can be both virtual or physical.Physical
- DPOD federated cell member members (FCMFCMs) must be installed in Non-appliance Mode with High_20dv architecture can be one of the following:
- Physical servers installed in Non-appliance Mode with High_20dv architecture type, as detailed in the Hardware and Software Requirements.
Physical servers are used when the cell is required to process high levels of transactions per second. - Virtual
- servers installed in Non-appliance Mode with Medium architecture type or higher, as detailed in the Hardware and Software Requirements
- .
- Virtual servers are used when the cell is required to process moderate levels of transactions per second, or when the cell is part of a non-production environment where the production cell uses physical servers (to keep environments architecture similar).
- Physical servers installed in Non-appliance Mode with High_20dv architecture type, as detailed in the Hardware and Software Requirements.
- A cell environment must have only physical or only virtual cell members (cannot mix physical and virtual cell members in the same cell).
- All DPOD federated cell members must have the same resources, such as CPUs, RAM, disk type and storage capacity.Each cell component
- (manager / FCMPhysical federated cell members with NVMe drives require special disks and mount points configuration to ensure performance. See Cell Member Configuration for NVMe Disks.
- Each cell component (manager / FCM) should have two network interfaces:
- External interface - for DPOD users to access the Web Console (on the cell manager) and for communication between DPOD and Monitored Gateways (on both the cell manager and the members).
- Internal interface - for internal DPOD components inter-communication (should be a 10Gb Ethernet interface).
- Network ports should be opened in the network firewall as detailed below:
...
From | To | Ports (Defaults) | Protocol | Usage | |||||
---|---|---|---|---|---|---|---|---|---|
DPOD Cell Manager | Each Monitored Device | 5550 (TCP) | HTTP/S | Monitored device administration management interface. | |||||
DPOD Cell Manager | DNS Server | TCP and UDP 53 | DNS | DNS services. Static IP address may be used. | |||||
DPOD Cell Manager | NTP Server | 123 (UDP) | NTP | Time synchronization | |||||
DPOD Cell Manager | Organizational mail server | 25 (TCP) | SMTP | Send reports by email | |||||
DPOD Cell Manager | LDAP | TCP 389 / 636 (SSL). TCP 3268 / 3269 (SSL) | LDAP | Authentication & authorization. Can be over SSL. | |||||
DPOD Cell Manager | Each DPOD Federated Cell Member | 443 (TCP) | HTTP/S | Communication (data + management) | |||||
DPOD Cell Manager | Each DPOD Federated Cell Member | 9300-9305 (TCP) | ElasticSearch | ElasticSearch Communication (data + management)DPOD Cell Manager | Each DPOD Federated Cell Member | 22 (TCP) | TCP | SSH root access is needed for the cell installation and for admin operations from time to time. | |
NTP Server | DPOD Cell Manager | 123 (UDP) | NTP | Time synchronization | Each Monitored Device | DPOD Cell ManagerEach DPOD Federated Cell Member | 9300-9305 (TCP) | ElasticSearch | ElasticSearch Communication (data + management) |
DPOD Cell Manager | Each DPOD Federated Cell Member | 60000-60003 (TCP) | TCP | SYSLOG Data | Each Monitored Device | DPOD Cell ManagerSyslog keep-alive data | |||
DPOD Cell Manager | Each DPOD Federated Cell Member | 60020-60023 (TCP) | TCP | HTTP/S | WS-M | Payloadskeep-alive data | |||
NTP Server | DPOD Cell Manager | 123 (UDP) | NTP | Time synchronization | |||||
Users IPs | DPOD Cell Manager | 443 (TCP) | HTTP/S | DPOD's Web Console | |||||
Admins IPs | DPOD Cell Manager | 22 (TCP) | TCP | SSH | |||||
Each DPOD Federated Cell Member | DPOD Cell Manager | 443 (TCP) | HTTP/S | Communication (data + management) | |||||
Each DPOD Federated Cell Member | DPOD Cell Manager | 9200, 9300-9400 | ElasticSearch | ElasticSearch Communication (data + management) | |||||
Each DPOD Federated Cell Member | DNS Server | TCP and UDP 53 | DNS | DNS services | |||||
Each DPOD Federated Cell Member | NTP Server | 123 (UDP) | NTP | Time synchronizationNTP Server | |||||
Each Monitored Device | Each DPOD Federated Cell Member | 123 (UDP) | NTP | Time synchronization | Each Monitored Device | Each DPOD Federated Cell Member | 6000060000-60003 (TCP) | TCP | SYSLOG Data |
Each Monitored Device | Each DPOD Federated Cell Member | 60020-60023 (TCP) | HTTP/S | WS-M Payloads | |||||
Admins IPs NTP Server | Each DPOD Federated Cell Member | 22 123 (TCPUDP)TCP | NTP | SSH | DPOD Cell Manager | Each DPODTime synchronization | |||
Admins IPs | Each DPOD Federated Cell Member | 60000-60003 22 (TCP) | TCP | Syslog keep-alive dataSSH | DPOD
Cell Manager Installation
...
Cell Manager Installation
Prerequisites
...
Prerequisites
- Make sure to meet the prerequisites listed at the top of this page.
- Install the following software package (RPM) on the cell manager: bc
DPOD Installation
Install DPOD as described in one of the following installation procedures:
- For Appliance Mode, follow the procedure:Follow Appliance Installation.
For Non-appliance Mode, follow the procedure: Non-Appliance Installation. - During installation the user is , when prompted to choose the data disk type (SSD / non SSD). Choose , choose the cell members disk type (should be SSD) instead of the cell manager disk type.
- For Non-appliance Mode:
- Follow the procedure: Non-Appliance Installation During installation, since the cell manager has two network interfaces (see prerequisites section), the user is During installation, when prompted to choose the IP address for the Web Console. Choose , choose the IP address of the external network interface.
After DPOD installation is complete, execute the following operating system performance optimization script commands and reboot the server:
Code Block | ||||
---|---|---|---|---|
| ||||
sed -i 's/^NODE_HEAP_SIZE=.*/NODE_HEAP_SIZE="2G"/g' /etc/init.d/MonTier-es-raw-trans-Node-1 /app/scripts/tune-os-parameters.sh reboot |
...
- Make sure to meet the prerequisites listed at the top of this page.
- Install the following software package (RPM) on the cell member: bc
- The following software packages (RPMs) are recommended for system maintenance and troubleshooting, but are not required: telnet client, net-tools, iftop, tcpdump, pciutils, nvme-cli
DPOD Installation
- Use Non-appliance Mode and follow the procedure: Non-Appliance Installation
This installation (before the federation process is executed later) is similar to a standard All-In-One standalone DPOD installation.
In order for this installation to complete successfully, all prerequisites for DPOD installation should be met as described in the installation procedure, including the 3 disk drives requirement. - The four-letter Installation Environment Name should be identical to the one that was chosen during the Cell Manager installation.
- During installation, since the cell manager has two network interfaces (see prerequisites section), the user is prompted to choose the IP address for the Web Console. Choose the IP address of the external network interface.
After DPOD installation is complete, execute the following operating system performance optimization script and reboot the server:
Code Block | ||||
---|---|---|---|---|
| ||||
/app/scripts/tune-os-parameters.sh
reboot |
Configuring Mount Points of Cell Member before Federation
The cell member is usually a bare metal server with NVMe disks for maximizing server I/O throughput.
Each of the Store's logical nodes (service) will be bound to specific physical processor, disks and memory using NUMA (Non-Uniform Memory Access) technology.
Required information
The following table contains the list of OS mount points that should be configured along with additional information that must be gathered before federating the DPOD cell member to the cell environment.
Please copy this table, use it during the procedure, and complete the information in the empty cells as you follow the procedure:
...
Configuring Mount Points of Cell Member
The cell member is usually a bare metal server with NVMe disks for maximizing server I/O throughput.
Each of the Store's logical nodes (service) will be bound to specific physical processor, disks and memory using NUMA (Non-Uniform Memory Access) technology.
Required information
The following table contains the list of OS mount points that should be configured along with additional information that must be gathered before federating the DPOD cell member to the cell environment.
Please copy this table, use it during the procedure, and complete the information in the empty cells as you follow the procedure:
Store Node | Mount Point Path | Disk Bay | Disk Serial | Disk OS Path | PCI Slot Number | NUMA Node (CPU #) |
---|---|---|---|---|---|---|
2 | /data2 | |||||
2 | /data22 | |||||
2 * | /data222 | |||||
3 | /data3 | |||||
3 | /data33 | |||||
3 * | /data333 | |||||
4 | /data4 | |||||
4 | /data44 | |||||
4 * | /data444 |
* Lines marked with asterisk (*) are relevant only in case DPOD sizing team recommends 9 disks instead of 6 disks per cell member. You may remove these lines in case you have only 6 disks per cell member.
...
OS Configuration of Cell Member before Federation
Installing NUMA software
...
Federation
Installing NUMA software
Execute the following command:
Code Block | ||||
---|---|---|---|---|
| ||||
yum install numactl |
Configuring local OS based firewall
Most Linux-based OS use a local firewall service (e.g.: iptables / firewalld). Since the OS of the Non-Appliance Mode DPOD installation is provided by the user, it is under the user's responsibility to allow needed connectivity to and from the server.
Configure the local firewall service to allow connectivity as described in the prerequisites section at the top of this page.
Note |
---|
When using DPOD Appliance mode installation for the cell manager, local OS based firewall service configuration is handled by the cell member federation script. When using DPOD Non-Appliance mode installation for the cell manager, local OS based firewall service configuration should be done by the user in addition to configuring the local OS based firewall service configuration of the cell memeber. |
DPOD Installation
Install DPOD:
- Use Non-appliance Mode and follow the procedure: Non-Appliance Installation
This installation (before the federation process is executed later) is similar to a standard All-In-One standalone DPOD installation.
In order for this installation to complete successfully, all prerequisites for DPOD installation should be met as described in the installation procedure, including the 3 disk drives requirement. - The four-letter Installation Environment Name should be identical to the one that was chosen during the Cell Manager installation.
- During installation, when prompted to choose the IP address for the Web Console, choose the IP address of the external network interface.
After DPOD installation is complete, execute the following operating system performance optimization commands and reboot the server:
Code Block | ||||
---|---|---|---|---|
| ||||
yum install numactl |
Configuring local OS based firewall
Most Linux-based OS use a local firewall service (e.g.: iptables / firewalld). Since the OS of the Non-Appliance Mode DPOD installation is provided by the user, it is under the user's responsibility to allow needed connectivity to and from the server.
Configure the local firewall service to allow connectivity as described in the prerequisites section at the top of this page.
Note |
---|
When using DPOD Appliance mode installation for the cell manager, local OS based firewall service configuration is handled by the cell member federation script. When using DPOD Non-Appliance mode installation for the cell manager, local OS based firewall service configuration should be done by the user in addition to configuring the local OS based firewall service configuration of the cell memeber. |
Cell Member Federation
In order to federate and configure the cell member, run the following script in the cell manager once per cell member.
For instance, to federate two cell members, the script should be run twice (in the cell manager) - first time with the IP address of the first cell member, and second time with the IP address of the second cell member.
Important: The script should be executed using the OS root user.
Code Block | ||||
---|---|---|---|---|
| ||||
/app/scripts/tune-os-parameters.sh
reboot |
Cell Member Federation
In order to federate and configure the cell member, run the following script in the cell manager once per cell member.
For instance, to federate two cell members, the script should be run twice (in the cell manager) - first time with the IP address of the first cell member, and second time with the IP address of the second cell member.
Important: The script should be executed using the OS root user.
Code Block | ||||
---|---|---|---|---|
| ||||
### If NUMA is enabled: ### /app/scripts/configure_cell_manager.sh -a <internal IP address of the cell member> -g <external IP address of the cell member> ### If NUMA is disabled: ### /app/scripts/configure_cell_manager.sh -a <internal IP address of the cell member> -g <external IP address of the cell member> -n true ### If cell member is virtual: ### /app/scripts/configure_cell_manager.sh -a <internal IP address of the cell member> -g <external IP address of the cell member> -i virtual For example: /app/scripts/configure_cell_manager.sh -a 172.18.100.34 -g 172.17.100.33 |
...