...

When APM Integration is enabled, a single Syslog record (in JSON format) is sent for each gateway transaction to an external APM, log aggregator or any other Syslog server (e.g.: IBM APM, Splunk, ELK, etc.).

The Syslog record contains information about the transaction which is aggregated from several sources of information DPOD has.

The feature requires DataPower FW 7.6+.

Value to Customers

• This feature allows customers to easily display gateway information on their APMs or log aggregators such as IBM APM, Splunk or ELK. This isolates the customer from changes to DataPower's log structure and saves the need to parse Syslog records.
• Customers may link from the displayed transaction in their APM to DPOD's transaction details, in order to enhance troubleshooting efforts.
• DPOD customers can use this feature to externalize DPOD information for data warehouse purposes.
• DPOD customers can retain only the summarized transaction details instead of all log records. This will increase history retention time period and minimize storage requirementsGateway transactions can be displayed in the broader context of the entire transaction flow across the different components the APM is monitoring.
• The APM gains access to knowledge only DPOD has about the gateway transactions (imagine that DPOD can tell the APM why the transaction failed in case of an error because DPOD analyzes the gateway logs).
• Parsing and storing the JSON records which DPOD produces for each transaction takes much less storage space and parsing efforts than collecting the raw log records that the gateway is producing using log targets.
• APM users can quickly drill down to any gateway transaction in DPOD for further analysis using deep links.
• This feature may be used to export DPOD information for data warehousing purposes, allowing reports and dashboards in the organization's standard tools.

Transaction Record Structure

The following table describes the fields that are logged with this feature.

...

{
  "_index": "[logical-tran-compact_i3][0]",
  "_type": "wdpLogicalTransChild",
  "_id": "ea5ae3c55b45be5500056a13_348659",
  "_timestamp": "2018-07-11T08:22:45.457Z",
  "_version": 5,
  "_operation": "INDEX",
  "_source": {
    "deviceName": "1cb3a54303a9",
    "domainName": "Infra_Domain",
    "latencyElapsed": 2,
    "microSecTimestamp": "2018-07-11T11:22:45.313729+03:00",
    "microSecTimestampStart": "2018-07-11T11:22:45.313729+03:00",
    "microSecTimestampFinish": "2018-07-11T11:22:45.315558+03:00",
    "serviceType": "xmlfirewall",
    "serviceUri": "/UpdateWantedMenProfiles_WHSW/Service.asmx",
    "serviceUrl": "http://Infra.HA:2555/UpdateWantedMenProfiles_WHSW/Service.asmx",
    "srcNodeName": "NODE0",
    "isError": false,
    "isTechnicalError": false,
    "clientIp": "172.77.77.5",
    "serviceName": "WSS_Loopback.XMLFW",
    "transactionId": "348659",
    "transactionGlobalId": "ea5ae3c55b45be5500056a13",
    "timeZone": "+03:00",
    "docAddedTimeInMil": 1531297365329,
    "timeHHMMSS": "11:22:45",
    "timeInMil": 1531297365313,
    "aggRecordVersion": "7.6.0.0+"
  }
}

Feature enablement

For each syslog agent in the system perform the following:

1. Edit the file /app/flume/syslog_agents/conf/MonTier-SyslogAgent-nn/flume_syslog.conf
2. Change the following property to true instead of false:
   MonTier-SyslogAgent-nn.sinks.syslogElasticSinknnn.serializer.enableLogicalTx = true
   

Stop and start Syslog agentsTo enable/disable this feature, please execute the following script: app_logical_tran.sh