This is a tech preview feature introduced in v1.0.5. When used, a syslog record will be sent for each DataPower transaction.

The syslog record used is a json-formatted data object, containing information aggregated from several sources relating to the transaction.

The feature requires DataPower FW 7.6+

Value to Customers

This feature allows customers to easily display DataPower information on their APMs or log aggregator such as IBM APMs , Splunk or ELK. This removes the dependency on, and isolates the customer from changes to DataPower's log structure.
Customers may add a link to the syslog record, which redirects from each transaction into DPOD' transaction details, in order to enhance troubleshooting efforts.
DPOD customers can use this feature to externalize DPOD information for Data Warehouse purposes.
DPOD customers can retain summarized transaction details. This will increase history information retention time period and minimize storage requirements.

Transaction Record Structure

The following table describes the fields that are logged with this feature.

Field Name	Description	Possible Values
serviceType	Service type as defined in DataPower	mpgw,wsp,xml-firewall,b2bgw
transactionGlobalId	DataPower global transaction ID (GTID)	26 chars long
transactionId	DataPower transaction ID (TID)	long number
srcNodeName	The name of the DPOD node that captured the transaction
domainName	DataPower domain name where the transaction was executed	String
deviceName	DataPower device name	String
timeYearOnly	Year of transaction start	format YYYY
timeMonthNum	Month number of transaction start	1-12
timeDay	Day of month of transaction start	1-31
timeHHMMSS	Full time of transaction start	format HHMMSS where: HH: 00-23 MM: 00-59 SS: 00-59
timeHour	Hour of transaction start	00-23
timeMinute	Minute of transaction start	0-59
timeSecond	Second of transaction start	00-59
timeMicroSec	Microsecond of transaction start
timeZone	The time zone used to log transaction start	format +ZZZZ
timeInMil	Transaction start time since Epoch in milliseconds	long number
timeInMicroSec	Transaction start time since Epoch in microseconds
timeDayInYear	Day of year of transaction start	1-365
timeSecondInDay	Second in the Day of transaction start	0-86399
timeMinuteInDay	Minute in the Day of transaction start	0-3599
timeDayInWeek	Day in a week of transaction start	1-7. 1- Sunday, 7-Saturday.
microSecTimestamp	Timestamp format of the time the transaction started	YYYY-MM-DDTHH:mm:ss.SSSSSS+ZZZZ
aggRecordVersion	Estimated FW version of the DataPower that executed the transaction. (For internal use)	String
technicalServiceName	Service Name. Note: in WSP we are not currently providing an operation name	String
technicalErrorMessage	Error message relating to the transaction. This field will only be populated when the transaction completed with error.	String
isTechnicalError	Indication whether the transaction completed with errors	true/false
aggErrorCode	Error Code in DataPower	String
message	The Syslog line that DPOD discerned is most likely to reflect the error cause	String
aggIndTXError	Indication that information on error transaction was received	true/false
aggIndTXFinished	Indication that information on transaction completion was received	true/false
aggIndTXStarted	Indication that information on starting transaction arrived	true/false
aggFirstTxOfGtx	Indication on whether this is the first transaction in a group. In this case -there might be a following transaction logged with the same GTID	true/false
microSecTimestampFinish	For internal use	long
aggUuidGtidEpochSeconds	For internal use	long
docAddedTimeInMil	For internal use	long
WDPTutXUuidGtidTid	For internal use	long
WDPTutXUuidGtidDeviceId	For internal use	String
aggUuidGtidTimst	For internal use	String
microSecTimestampStart	For internal use	String

JSON Example

{
   "_index" : "logical-tran-compact_i1",
   "_type" : "wdpLogicalTrans",
   "_id" : "802d48ad5976a98f00080cc4_527556",
   "_timestamp" : "2017-07-25T02:15:23.279Z",
   "_version" : 4,
   "_operation" : "INDEX",
   "_source" : {
      "serviceType" : "mpgw",
      "timeDayInYear" : "206",
      "transactionGlobalId" : "802d48ad5976a98f00080cc4",
      "timeDayInWeek" : "2",
      "microSecTimestampStart" : "2017-07-25T05:14:39.570000+0000",
      "timeHHMMSS" : "05:14:39",
      "aggUuidGtidTimst" : "2017-07-25 02:14:39",
      "aggFirstTxOfGtx" : "true",
      "aggIndTXStarted" : "true",
      "deviceName" : "idg7600",
      "timeSecond" : "39",
      "aggUuidGtidEpochSeconds" : "1500948879",
      "timeSecondInDay" : "18879",
      "timeMinuteInDay" : "314",
      "timeYearOnly" : "2017",
      "timeInMicroSec" : "1500948879569000",
      "srcNodeName" : "NODE0",
      "timeDay" : "25",
      "microSecTimestamp" : "2017-07-25T05:14:39.569000+0000",
      "WDPTutXUuidGtidDeviceId" : "802d48ad",
      "timeInMil" : "1500948879569",
      "timeZone" : "+0000",
      "transactionId" : "527556",
      "timeMinute" : "14",
      "timeMonthNum" : "07",
      "domainName" : "APIMgmt_ACB198F9A6",
      "timeMicroSec" : "569000",
      "timeHour" : "05",
      "WDPTutXUuidGtidTid" : "00080cc4",
      "aggRecordVersion" : "7.6.0.0+",
      "technicalServiceName" : "GetDeliveryStatus_MHJV.MPGW",
      "docAddedTimeInMil" : "1500948882966",
      "technicalErrorMessage" : "Invalid JSON format",
      "isTechnicalError" : "1",
      "aggErrorCode" : "0x02130008",
      "message" : "<11>2017-07-25T05:14:39.570000+0000 MonTierLocalId-8 [0x02130008][mpgw][error] mpgw(GetDeliveryStatus_MHJV.MPGW): trans(527556)[error][192.168.0.112] gtid(802d48ad5976a98f00080cc4): Invalid JSON format\r",
      "aggIndTXError" : "true",
      "microSecTimestampFinish" : "2017-07-25T05:14:39.573000+0000",
      "aggIndTXFinished" : "true"
   }
}

Feature enablement

Perform the following steps to enable this feature

Install and configure Store plugin.
Configure each syslog agent.
Stop and start all system services.

Plugin Installation and Configuration

The plugin is located at /installs/tech-preview/es-changes-feed-plugin.zip
To extract the plugin:
1. cd /installs/tech-preview
2. unzip es-changes-feed-plugin.zip
3. The command will create the following files in the directory /installs/tech-preview :
  1. MonTierEventsFeedPlugin.zip
  2. MonTierEventsFeedPlugin.zip.md5
To install plugin execute the following commands:
1. cd /app/elasticsearch_base
2. bin/plugin install file:///installs/tech-preview/MonTierEventsFeedPlugin.zip
3. Approve the installation by pressing Y to the prompt question "Continue with installation? [y/N]"
4. To remove plugin issue command : bin/plugin remove MonTierEventsFeedPlugin
Configure each Store node number 2 or 4 as follow:
1. cd /app/elasticsearch_nodes/config/MonTier-es-raw-trans-Node-2
2. Edit the elasticsearch.yml file

add to the end of file the parameters:

Parameter	Values	Description
montier.events.feed.syslog.protocol	tcp or udp	the protocol used to send messages to syslog
montier.events.feed.syslog.host	ip address v4 or valid hostname	target syslog agent hostname
montier.events.feed.syslog.port	integer 1-65535	target syslog agent port
montier.events.feed.syslog.ssl	false or true	disable or enable SSL
montier.events.feed.appname	text A-Z\|a-z\|0-9 name - 8 chars	syslog application name

Configure syslog agent

For each syslog agent in the system perform the following:

cd /app/flume/syslog_agents/conf/MonTier-SyslogAgent-nn/ (replace nn with the agent number)
edit the flume_syslog.conf file
Locate in each agent rows with pattern: MonTier-SyslogAgent-nn.sinks.syslogElasticSinknnn.batchSize = (replace nnn with sink name 1 to 10. Also - replace nn with the agent number)
Add below each of these rows the line below (substituting nn and nnn as described in step 3 above)
MonTier-SyslogAgent-nn.sinks.syslogElasticSinknnn.serializer.enableLogicalTx = false

Browser not supported