Components Diagram

The diagram below depicts DPOD's components and their interactions.

(Note: DPOD was designed for deployment using either a standalone (All in one) or distributed topology. When distributed topology is used, some of the components above will reside on different virtual/physical servers.)

Big Data Store

The data flowing into DPOD through the stream processing component is stored in a Big Data store.

A new DPOD installation sets up a Big Data store that is preconfigured and optimized for DPOD's requirements. As the storage is managed automatically by DPOD’s internal components, no regular maintenance is required by users.

Configuration Database

DPOD uses an internal relational database to store its system configuration. DPOD's components read configuration properties from this database.
Configuration data is updated via the console (web interface), and is accessible by web interface and several internal components.
DPOD also stores report templates and scheduling information in this database.

Log Collection agents (syslog agents)

The log collection agents receive Syslog log records from the monitored devices' Syslog log target, which is configured by DPOD during initial system configuration. The agents then parse the log entries, store them in the Big Data store and send them to the stream analyzing component.

DPOD automatically creates a log target on each domain (including the default domain).

Each monitored device's log target can send logs to a single log collection agent. However, a single log collection agent may subscribe to Syslog targets originating from multiple domains on a single device or several domains on different devices.

Using DPOD's user interface, a DPOD system administrator may tune the links between Syslog targets and specific log collection agent (based on the Syslog target's logging rate). Doing so helps balance the network transmissions across the system and enhance throughput and log processing on DPOD's side.

During DPOD's installation process, the installer gathers environment planning and sizing request data. Based on this information, the installer creates and configures a number of log collection agents.

Sampling Agents (Device Resources and Service Resources)

DPOD’s sampling agents use DataPower's SOMA (SOAP configuration Management) interface for retrieving operational data for both devices (e.g. CPU, memory, load, file system and sensors) and services (e.g. service memory, service configuration). Sampling records are stored in the Big Data store.

The sampling interval used may be configured. The default sampling interval is 30 seconds for device data and 300 seconds for service data and can be changed.

WS-Management Agents (WS-M)

The DPOD WS-M agents, using the WS-Management agent on monitored devices, are responsible for processing and storing the monitored devices' service payload recording from WS-Proxy services,

Payload recording is off by default, and has to be enabled manually through DPOD's user interface. For security reasons, enabling the WS-M Agents in IDG is manual as well. Once this is done, a subscription is registered in the WS-M agent on the monitored device.

Once a payload is recorded by a monitored device's WS-M agent from a WS-Proxy policy, the data is pushed to DPOD’s WS-M agent, which stores it in the Big Data store.

Payload recording puts the system under substantial load and therefore has to be manually enabled and its duration is limited up to 60 minutes.

Stream Processing

DPOD uses the Stream Processing component to stream, parse and analyze incoming data. It collects data from the log collection and WS-M agents, and operates automatically without requiring maintenance.

User Interface (Web Console)

DPOD's user interface (Web Console) is a web-based user interface. The Console can be accessed via HTTPS and requires user/password authentication.

The console enables the user to troubleshoot, analyze and gain insights into transaction activity on monitored devices. Privileged users may also use it to update the system configuration.

Reports

DPOD allows users to generate various reports. These reports may be run on an ad-hoc or scheduled basis. The Reports component is responsible for processing and generating these user reports.

The system is installed with a number of built-in reports (e.g. Services Elapsed Time, System Errors and Service Memory), and privileged users are able to configure new custom reports via DPOD’s Web Console.

Reports can be saved as CSV files on the DPOD appliance file system, or sent as mail attachment via SMTP and a custom web service.

Wiki Portal

DPOD creates a service wiki on the fly, according to configuration changes and activity data which it collects. The process creates new wiki entries for newly created services, and updates existing entries when existing services are changed. Data in the wiki may be sourced from DataPower configuration, activity logs or the user themselves. Both WS-Proxy (WSP) and Multi-Protocol Gateway (MPGW) services are documented in the wiki. See Update Wiki service JSON for more information.

Browser not supported