Component Diagram
The diagram below depicts DPOD's components and their interactions.
...
(Note: DPOD was designed for deployment using either a standalone (All in one) or distributed topology. When distributed topology is used, some of the components above will reside on different virtual/physical servers.)
Big Data Store
The data flowing into DPOD through the stream processing component is stored in a Big Data store.
A new DPOD installation sets up a Big Data store that is pre-configured and optimized for DPOD's requirements. As the storage is managed automatically by DPOD’s internal components, no regular maintenance is required by users.
Configuration Database
DPOD uses an internal relational database to store its system configuration. DPOD's components read configuration properties from this database.
Configuration data is updated via the console (web interface), and is accessible by web interface and several internal components.
DPOD also stores report templates and scheduling information in this database.
...
DPOD’s sampling agents use DataPowerGateway's SOMA (SOAP configuration Management) interface for retrieving operational data for both devices (e.g. CPU, memory, load, file system and sensors) and services (e.g. service memory, service configuration). Sampling records are stored in the Big Data store.
The sampling interval used may be configured. The default sampling interval is 30 seconds for device data and 300 seconds for service data and can be changed.
WS-Management Agents (WS-M)
The DPOD WS-M agents, using the WS-Management agent on monitored devices, are responsible for processing and storing the monitored devices' service payload recording from WS-Proxy services,
Payload recording is switched off by default, and has to be enabled manually through DPOD's user interface. For security reasons, enabling the WS-M Agents in IDG Gateway is a manual process as well. Once this is done, a subscription is registered in the WS-M agent on the monitored device.
...
Payload recording puts the system under substantial load and therefore has to be manually enabled and its maximum duration is limited to 60 minutes.
Stream Processing
DPOD uses the Stream Processing component to stream, parse and analyze incoming data. It collects data from the log collection and WS-M agents, and operates automatically without requiring maintenance.
User Interface (Web Console)
DPOD's user interface (Web Console) is a web-based user interface. The Console can be accessed via HTTPS and requires user/password authentication.
The console enables the user to troubleshoot, analyze and gain insights into transaction activity on monitored devices. Privileged users may also use it to update the system configuration.
Reports
DPOD allows users to generate various reports. These reports may be run on an ad-hoc or scheduled basis. The Reports component is responsible for processing and generating these user reports.
...
Gateway Maintenance Activities
A maintenance activity defines the set of maintenance actions required for a specific goal. An example of such an activity is "Perform Secure Backup on device X".
Additionally, the maintenance activity contains other specific definitions for the action. This may include for instance which certificate should be used for the Secure Backup or which deployment policy should be used for a configuration sync operation.
Current provided activities are: backup and configuration sync.
DPOD allows users to define a plan that includes a set of target gateways on which activity will be performed and a receipt on how to perform the activities.
These Activities can be integrated into an organisation wide DevOps process by REST invocation.
Alerts
DPOD can publish alerts when certain predefined events occur, for example, when device CPU is over 80%. Alerts can be viewed and managed from the the Alerts Setup page.
An alert consists of:
Alert Query - the metadata that defines the alert parameters (for example, count all the system errors occured occurred in the last 10 minutes in domain DMZ). There are several builtin built-in queries. Users customized threshold is a pramater parameter in the query.
Alert Execution - one execution of the alert query
...
Transactions Event Feeder
This component handle the creation and publishing of a single aggrgated logical aggregated logical transaction record.
One of the common usage is to push flat transactional data to external system or centralized data to have a flexiable and flexible and easy acces access with analyticstools.
DevOps Services Portal
DPOD now provides a new self service DevOps portal for -service DevOps Services Portal for traditional services. This is a new portal dashboard where end-users can:
Execute two new actions on SOAP Web services
withwith local WSDL: validate and promote
The Validate action uploads your new WSDL and schema files to a temporary location, validates its compilation and creates a temporary WS-Gateway to ensure the object is up.
Promote action allows uploading a new WSDL and schema files to the target location (device and domain) and creating a new version of the service.
Execute two new actions on SOAP Web services
withwith remote WSDL: validate and promote
The Validate action updates a new or existing URL of a remote WSDL, validates its compilation, and creates a temporary WS-Gateway to ensure the object is up.
The Promote action updates a new or existing URL of a remote WSDL in a target location (Device and Domain), and creates a newer version of the service.
All actions require permissions set by a security policy (custom roles).
Each action may be extended or customized using Python scripts. Example scripts are open source and may be obtained from a git repository - see docs documentation.
Task Scheduling Mechanism
The task scheduling mechanism consists of a scheduler and several workers.
In order to have a scalable and resilient execution of tasks, the scheduler assigns tasks to multiple workers.