IBM DataPower Operations Dashboard v1.0.16.0
A newer version of this product documentation is available.
You are viewing an older version. View latest at IBM DPOD Documentation.
What's New
The features listed below were added to DPOD. The document's icon conventions are listed at the bottom.
New Features in 1.0.16.0
Improved User Experience
A new dashboard for API activity by time interval shows statistics and detailed data of the API calls per minute/hour/day/etc.
Security
Custom roles may be used to allow users of any builtin role to access Expert Mode.
Custom roles may be used to allow users of any builtin role to view and execute report and alerts.
Sensitive data may be redacted based on patterns, to prevent DPOD from storing and displaying sensitive data.
Upgrade Assistant
A new Upgrade Assistant prepares the DPOD instance for upcoming upgrades.
Data is reindexed in the background (if required) while DPOD is up and running.
View the status of the reindexing and configure its behavior through a new page under the Manage section.
This is replacing the Data Migration Tool, that was previously used during upgrades to reindex data.
REST APIs
A new set of REST APIs for managing and executing reports.
A new set of REST APIs for managing and executing alerts.
New Features in 1.0.15.1
Important security fixes.
New Features in 1.0.15.0
Improved User Experience
Custom roles may be used to limit user access to product views (Gateway / API-C).
Custom roles may be used to allow users of any role to edit report and alerts.
Reports and alerts are displayed per product view to easily distinguish between API Connect and DataPower Gateway services.
All tables now support the new context menu which allows easier filtering and copy.
Quick setup modal after installation/upgrade allows choosing the default product view and theme.
API Connect Integration
API Connect Analytics Offload may be used to send payload and request/response headers to DPOD.
When activating API Connect Analytics Offload, payload and HTTP headers are displayed per transaction and are available for filtering.
Expert Mode provides access to the data that was captured from API Connect Analytics Offload with the ability to create custom dashboards.
Cleaner view of the assembly flow (toggle between primary policies and all policies).
Performance
Support for enhanced Syslog log target performance introduced in DataPower Gateway firmware.
New Features in 1.0.14.0
Improved User Experience
Many filters (including device, domain, service, API name, product, etc.) support wildcards (*) for matching.
Filters across the system support operators (including reports and alerts).
A new context menu in main tables (transactions, early failing, raw messages, etc.) allows easier filtering and copy.
Custom dashboards may be used as the home page.
Some widgets that could not be added to custom dashboards, such as Transactions List, are now supported.
CSV export size has been increased to 2,500 records instead of 1,000.
Any alert and report can now be duplicated, no matter if they arrive out-of-the-box or created from scratch.
API Connect Integration
Provider organization and consumer organization can be used in custom roles to limit user access.
Many filters have been added to API Connect dashboards, including provider organization, HTTP response code, HTTP method, elapsed time, request/response size, etc.
Added Top Consumers table in Recent Activity dashboard.
Status column in API Connect Transactions page was replaced by HTTP response code column.
Transaction error analysis now detects some more common error scenarios with APIC v10.
Deprecated Extended Transactions
The deprecated Extended Transaction functionality, which required instrumentation of the processing policy, and was replaced in 1.0.8.0 by a non-intrusive alternative, has been removed.
New Features in 1.0.13.0
Improved User Experience
Quickly move to next and previous transactions from the transaction page.
Home page under User Preferences has many more options.
API Connect Integration
Added support for Web Socket Upgrade policy (requires gateway firmware 10.0.2+).
Supported Platforms
Added support for RHEL 8.3 (non-appliance installation only).
New Features in 1.0.12.0
Improved User Experience
Added filter operators, such as "Equal", "Not Equal", "Greater Than" and "Lower Than" to all dashboards.
Improved Time filter - add previous and next time period buttons, add new options and allow to manually edit custom time.
New Report: Service Request/Response Size Stats.
Gateways table is now sorted by name.
New Task Scheduling Mechanism
New task scheduling mechanism enhances the system stability and improves the response time of tasks including reports and alerts.
New Scheduler Info page under the Manage menu displays information about the scheduled tasks.
New Features in 1.0.11.0
Correlated Transactions Tab
New tab in the transaction drill-down page, showing a list of all other transactions with the same global transaction ID.
The list shows both APIC and Gateway transactions.
Raw Messages over Time Dashboard
Show amount of raw messages over time.
Show total amount of raw messages grouped by message codes.
API Connect Integration
APIC transactions status is solely determined by their front-end response code.
Improved User Experience
New user preference to enable dependency between device, domain and service filters.
New Alert: API Error Message Count Alert.
New Alert: Trans. Over 30 sec by Service Name Alert.
New Alert: Unavailable Devices Alert.
New Report: Top Transactions Count per Interval.
New Report: Top Documents Count per Interval.
New Report: Top Documents Count per Agent per Interval.
New Report: Service Latency Report - Extended.
New Report: Service and URI Total Transactions Report.
Supported Platforms
Added support for Gateway/APIC v10
New Features in 1.0.10.1
Important security fixes.
New Features in 1.0.10.0
API Connect Integration
Support for DataPower API Gateway.
Extended transactions page now shows API Connect transactions (when extended transactions are enabled).
Reworked API Connect transaction drill-down page to show more accurate transaction flow and latency information.
LDAP Integration
Improved integration with LDAP to allow more flexibility and simplicity in the LDAP registry configuration.
Improved User Experience
New report - latency statistics grouped by gateway/domain/service.
Reports and Alerts - Support TLSv1.2 for SMTP with STARTTLS.
New gateway information panel.
New internal alert on misconfigured DPOD log targets.
Better support for IDG 2018 SOMA error messages.
Speed up Failed Objects sampling to make alerts more reliable.
API Connect API Policies dashboard is now sorted by API Name and Policy Name.
API Connect URI and URL calls dashboards can now be filtered by API attributes.
New Features in 1.0.9.0
Appliance Migration Assistant
A step by step wizard to backup and restore gateway configuration in order to migrate from one gateway to another.
The migration assistant backs up a source gateway, performs validation on both source and target, and restores the backup to the target gateway.
Services Configuration Changes Analysis
A new dashboard showing all recent service configuration changes and their effect on latency and on the number of errors.
The transaction drill-down page indicates whether a recent configuration change was made to the underlying service.
Reports and Alerts Improvements and New History Dashboards
Reports and alerts executions are stored and can be downloaded from new history dashboards.
More filter types are available for API Connect reports and alerts, as reports and alerts are now bound to a product (Gateway or API Connect).
API Connect Integration Improvements
Added context variable capturing to the policy variables capture.
Handle more cases of early failed transactions (OAuth, 403, etc).
Improved error analysis and payload recording.
New Admin Interface
Most of the system's admin functionality, such as stopping or starting services, is available through a new web console.
Improved User Experience
New dashboard showing stats of gateway's MQ objects, such as Active and Idle connections.
Added a new alert and a system health metric for MQ Connections Utilization.
Added a new alert for monitoring services that did not run in the last X minutes.
Added a new alert for API Connect transactions that ended with a specific error code or with an error.
Added a new Transactions Latency Statistics report.
Added support for Sync and Firmware upgrade for IDG 2018.x firmwares.
New Features in 1.0.8.6
Custom Dashboards
Admins can design custom dashboards containing DPOD widgets.
Custom dashboards are available to all users from the Dashboards menu.
API Connect Integration Improvements
New option to trace and view policy variables for specific API Connect transactions (by Catalog and API Name).
Improved API Connect Side Calls analysis.
Support for Monitoring Transactions in the Default Domain
DPOD now supports monitoring transactions in the default domain.
Improved User Experience
When removing a monitored device from DPOD - all DPOD's log targets can be removed from the monitored device.
New Features in 1.0.8.5
API Connect Integration Improvements
Many new dashboards in the API-C product view, including APIs activity, latency, HTTP response codes and consumers.
New dashboard for comparing latency between API versions.
New dashboard for comparing API assembly policies between APIs or versions.
Several filters have been added to API-C pages with auto-complete, such as catalog, space, product, plan, API name, API version and consumer application name.
Detection of API Connect early failed transactions improved significantly.
Automation Scripts for Creating Federated Architecture
DPOD's processing can now be distributed across different federated installations in order to handle very high transaction loads (thousands of transactions per second).
DevOps Services Portal Improvements
New functionality to import a service into multiple devices.
New functionality to view and filter services by user comments.
Request and Response Size (requires firmware 7.6.0.8+ or 7.7.1.2+)
In the Transactions list page, results can be filtered using request and response size of the transactions.
Request and response sizes are displayed in the transaction details page.
Response size has been added to Activity Distribution dashboard.
Improved User Experience
Yesterday, Last Week and Last Month added to the time filter.
Raw messages can now be filtered by From/To B2B Partner Profiles.
Reports and alerts queries can have named parameters.
New reports: average service latency, average request size and services above specific elapsed time.
New alert: message code threshold.
Backup, Sync and Firmware upgrade plans can now be canceled before they start to execute.
Supported Platforms
Added support for RHEL 7.5.
Upgraded Java run-time and Store versions.
New Features in 1.0.8.0
System Health
An overall health overview for all gateways according to user defined metrics.
Drill-down to view the device's health in the last 60 minutes.
Non-Intrusive Extended Transactions ( requires firmware 7.7.1.1+)
Extended Transactions, which are transactions that span more than one monitored devices/services, are now being tracked in a non-intrusive way over HTTP protocol, without the need for processing policy instrumentation.
The pre v1.0.8, intrusive Extended Transaction functionality will be deprecated in the near future.
API Connect Integration Improvements
Support for API Connect version 2018.2.5 compatibility mode.
New DPOD Roles to allow and revoke access to API Connect field such as API Name, Space and Plan names.
API Connect views can now be set as the default system view.
Service URL/URI Calls dashboards are now available in API Connect view.
Performance and History
User can choose between supporting more TPS but gathering less data per transaction, or collecting more data and supporting less TPS. This option is available during log target setup.
APM Integration
Providing a publisher to Splunk with a Splunk application as a reference implementation for the integration (requires firmware 7.7.1+).
Publishing more analytical data fields.
Simplify setup of implementing APM integration.
DevOps Services Portal Improvements
New functionality to stop or start a service based on custom role permissions.
Docker Support
Publish DPOD developer edition image in Docker Hub.
Improved User Experience
All dashboards can be exported to a CSV file.
Domain filter now support asterisk wildcard.
New REST API to create WS-M subscriptions.
Device Resources page now shows Ethernet Interfaces statistics.
New alert on more than 5 API Connect APIs with latency above X ms.
Provide retry functionality for failed publishing via SMTP or syslog of alerts, internal health and maintenance jobs.
Admins can use a new page to delete transactions logs that are stored in DPOD.
New Features in 1.0.7.0
Tenant Support
Added the ability to monitor transactions information on appliance with Tenants.
Resource monitoring and appliance maintenance are not available for tenants.
API Connect Integration
New API Connect Recent Activity dashboard that displays a high-level overview of APIs activity.
The Early Failing Requests Dashboard in now also available in API Connect view.
Alerts can be generated based on some of APIs latency data.
Auto Setup Domains feature detects a Domain Reset to auto-setup it again.
Maintenance Plans Improvements
DPOD now provides the ability to upgrade virtual and physical appliance firmware (requires firmware 7.5+) - see limitations
Maintenance Plan for Syncing DataPower Appliance Configuration is now supported (initially announced as tech preview from v1.0.5)
Improved User Experience
DataPower and API Connect transactions lists may be exported to a CSV file.
The transaction error analysis now indicates whether a probe was enabled during its execution.
New latency alerts based on percentile.
Added latency percentile data to Service URL/URI Calls dashboards.
Payload capture can be enabled on multiple devices or domains at once.
DevOps Services Portal - allow to setup a separate temporary domain (for WSDL validation and promotion) for each device.
Columns can be hidden or reordered in the Transactions, Raw Messages and API Connect Transactions pages.
New Internal Monitoring page that shows DPOD's components status and recent alerts to assist monitoring DPOD internal health.
New Features in 1.0.6.0
DevOps Services Portal
DPOD now provides a new self service DevOps portal for traditional services. This is a new portal dashboard where end-users can:
Execute two new actions on SOAP Web services with local WSDL: validate and promote
The Validate action uploads your new WSDL and schema files to a temporary location, validates its compilation and creates a temporary WS-Gateway to ensure the object is up.
Promote action allows uploading a new WSDL and schema files to the target location (device and domain) and creating a new version of the service.
Execute two new actions on SOAP Web services with remote WSDL: validate and promote
The Validate action updates a new or existing URL of a remote WSDL, validates its compilation, and creates a temporary WS-Gateway to ensure the object is up.
The Promote action updates a new or existing URL of a remote WSDL in a target location (Device and Domain), and creates a newer version of the service.
All actions require permissions set by a security policy (custom roles).
Each action may be extended or customized using Python scripts. Example scripts are open source and may be obtained from a git repository - see docs
Early Failing Requests dashboard
Added a new dashboard for early failing requests (such as TLS/SSL failures) - where a request is received but is unable to reach the execution phase of a Processing Rule of a Service - (requires firmware 7.5.2.4+)
Service API Dashboard
DPOD now provides a new dashboard that provides statistics pertaining to service URIs. These statistics include the number of invocations and min/max/avg latency in addition to the Service API (URL) Dashboard
Improved User Experience
Users can now save favorite filter combinations and use them in subsequent sessions.
A new free text search field was added to the Transactions List page and Raw Messages Page where the user may enter a single value that will be searched across various data fields.
Service filter now supports auto completion.
All applicable filters support multi-select of values.
Users can now change the size of the series (number of values) that is displayed in charts of type "top N".
A recent notifications panel was added to The Navigation Bar. It displays the recent UI notifications for the user (not persisted).
A new "Share" button added. It allows sharing the current page (including filters, transaction ID etc.) by copying/pasting the current URL or by sending it via email
Time range filters now provides more options: last 10 minutes, start of day, start of week, this month.
Consistent behavior across pages of zoom into a transaction or use its value as a filter criteria.
Data Migration Procedure
From time to time, DPOD customers may be required to install a new DPOD installation while preserving the data already collected. for example:
Customer deployed DPOD Appliance mode at version v1.0.0 with CentOS 6.7 and now requires to migrate to CentOS 7.2 introduced at version v1.0.2+.
Customer deployed DPOD on a virtual server but load increased and now requires a physical server.
Customer deployed DPOD Appliance mode but wants to migrate to Non-Appliance mode (RHEL) to better comply with their organization security requirements and standards.
New procedure and tools were introduced to support customers with migration of an existing DPOD Store data to a new DPOD installation in each of the scenarios above.
New Features in 1.0.5.0
API Connect Integration
DPOD now has new capabilities when coupled with API Connect v5.0.7.2+ (API-C) and DPOD
Display a dedicated transaction list with only API-C transactions
Filter by API-C fields (e.g. API name, API version, catalog, HTTP method, API urls and a dozen more fields)
Calculate and display latency graphs for each policy execution inside an API (including custom policies)
Provide both API-C and DataPower views of the same transaction, and the ability to switch between them
A new dashboard of APIs and the date they were last executed in the system.
API payloads may now be captured
Automatically run setup DPOD log targets for each new API-C domain created. (This requires enablement in the monitored device configuration). Further configuration based on domain name prefix is available.
To activate API-C integration, perform (from the Web Console) a re-configuration of all your log targets in the DataPower that runs the API-C domain. You can also use the auto-setup feature to handle API-C domains created in the future.
Maintenance Plan for Syncing DataPower Appliance Configuration () - see limitations
DPOD now provides a multitude of syncing capabilities:
Sync a single source configuration (domains/devices from a single appliance) to multiple targets (domains/devices) accordingly. Wildcard patterns supported.
A script may be executed as part of the sync lifecycle - before running, on completion and before/after each single sync.
Periodic sync task scheduling
The domain or device may be quiesced before execution
Configuration sync with password map alias, provided that phrase and all objects are already at target and completely identical (firmware 7.5.2.4 and above)
To use this feature, a remote file system for temporary storage is required. Ensure to read and understand this feature's limitations, as it disables runtime while syncing
Maintenance Plan for DataPower Backup and Appliance Configuration - see limitations
DPOD now provides a multitude of backup capabilities:
Backup a group of devices/domains. Wildcard patterns supported.
Domain or device export or secure backup
Ability to quiesce the domain or device before execution
A script may be executed as part of the backup lifecycle - before running, on completion and before/after each single backup
To use this feature, a remote file system for temporary storage is required. Ensure to read and understand this feature's limitations, as it disables runtime while performing the backup
Side Call Analysis - firmware 7.6+
DPOD now provides latency analysis for calls performed within a transaction using protocols such as HTTP, MQ, LDAP, ODBC.
DPOD now provides details of the order of execution for these side calls.
To activate Side Call Analysis on DPOD run re-configuration of all your log targets in the DataPower that runs the API-C domain.
External APM Integration () - firmware 7.6+
DPOD now provides the ability to push transaction aggregated data as one JSON record to external APMs (such as Tivoli APM) or log aggregators (such as Splunk).
To activate this feature, a series of manual configuration steps will need to be performed, and a system shutdown will be required.
Service API Dashboard
DPOD now provides a new dashboard that shows statistics pertaining to a service URL, such number of invocations and min/max/avg latency.
Introducing DPOD environment
A monitored device may now be attached to several DPOD installations.
Naming each DPOD installation and reflection of this.
Node naming and configuration editing from the UI.
During installation / upgrade you will need to set the system variables.
Feature Improvements
New alert for transaction errors
New report: export basic monitored device service configuration
Improved High Availability Scenarios
A limited Active / Active scenario is now supported if the same monitored device is connected to two separate and independent DPOD installations
New REST APIs introduced, to support Active / Standby scenario
New filter to search logs by B2B message id or B2B partner id
Improved error Analysis by providing simple human readable errors that are most common in:
SSL scenarios within a transaction
Side calls scenarios within a transaction (when the Side Call feature is enabled)
API Connect scenarios within a transaction (when the API-C feature is enabled)
Deprecated features in future versions
The Extended Transaction feature is expected to be deprecated for customers using firmware above 7.6. This feature will be replaced using a Non-Intrusive technique in upcoming releases.
The following conventions are used in this document:
- Some actions are required to enable the feature.
- This is a Tech Preview feature. Consult the release notes for more information.