In v1.0.5 a new feature was introduced that send syslog record for each datapower transaction.

This syslog record is a json formatted data object the consist of information aggregated from several sources on the transaction.

The feature required DataPower FW 7.6+

Customer value

• Feature assist customers to easily show datapower information on their APMs or log aggregator such IBM APMs , Splunk or ELK without being exposed to changes in DataPower log structure.
• Customer can also add a link to redirect from each transaction into DPOD details transaction and enhance troubleshooting
• DPOD customer scan externalize DPOD info for Data Warehouse purpose .
• DPOD customers can keep summarize transaction details . This will increase history time and minimize storage.

Transaction Record structure

 JSON example:

{
"_index" : "logical-tran-compact_i1",
"_type" : "wdpLogicalTrans",
"_id" : "802d48ad5976a98f00080cc4_527556",
"_timestamp" : "2017-07-25T02:15:23.279Z",
"_version" : 4,
"_operation" : "INDEX",
"_source" : {
"serviceType" : "mpgw",
"timeDayInYear" : "206",
"transactionGlobalId" : "802d48ad5976a98f00080cc4",
"timeDayInWeek" : "2",
"microSecTimestampStart" : "2017-07-25T05:14:39.570000+0000",
"timeHHMMSS" : "05:14:39",
"aggUuidGtidTimst" : "2017-07-25 02:14:39",
"aggFirstTxOfGtx" : "true",
"aggIndTXStarted" : "true",
"deviceName" : "idg7600",
"timeSecond" : "39",
"aggUuidGtidEpochSeconds" : "1500948879",
"timeSecondInDay" : "18879",
"timeMinuteInDay" : "314",
"timeYearOnly" : "2017",
"timeInMicroSec" : "1500948879569000",
"srcNodeName" : "NODE0",
"timeDay" : "25",
"microSecTimestamp" : "2017-07-25T05:14:39.569000+0000",
"WDPTutXUuidGtidDeviceId" : "802d48ad",
"timeInMil" : "1500948879569",
"timeZone" : "+0000",
"transactionId" : "527556",
"timeMinute" : "14",
"timeMonthNum" : "07",
"domainName" : "APIMgmt_ACB198F9A6",
"timeMicroSec" : "569000",
"timeHour" : "05",
"WDPTutXUuidGtidTid" : "00080cc4",
"aggRecordVersion" : "7.6.0.0+",
"technicalServiceName" : "GetDeliveryStatus_MHJV.MPGW",
"docAddedTimeInMil" : "1500948882966",
"technicalErrorMessage" : "Invalid JSON format",
"isTechnicalError" : "1",
"aggErrorCode" : "0x02130008",
"message" : "<11>2017-07-25T05:14:39.570000+0000 MonTierLocalId-8 [0x02130008][mpgw][error] mpgw(GetDeliveryStatus_MHJV.MPGW): trans(527556)[error][192.168.0.112] gtid(802d48ad5976a98f00080cc4): Invalid JSON format\r",
"aggIndTXError" : "true",
"microSecTimestampFinish" : "2017-07-25T05:14:39.573000+0000",
"aggIndTXFinished" : "true"
}
}

Feature enablement

To enable this feature 3 steps are required: 

1. Install and configure Store plugin.
2. Configure each syslog agent.
3. restart system
   
   

Plugin install and configure

1. The plugin is located at  /installs/tech-preview/es-changes-feed-plugin.zip
2. In order to extract the plugin please follow the procedure :
   1. cd  /installs/tech-preview
   2. unzip es-changes-feed-plugin.zip
   3. The command will create the following files in the directory /installs/tech-preview :
      1. MonTierEventsFeedPlugin.zip
      2. MonTierEventsFeedPlugin.zip.md5

3. To install plugin just run commands: 

   1. cd /app/elasticsearch_base

   2. bin/plugin install file:///installs/tech-preview/MonTierEventsFeedPlugin.zip

   3. Approve the installation by pressing Y to the prompt question "Continue with installation? [y/N]"

   4. To remove plugin issue command : bin/plugin remove file:///installs/tech-preview/MonTierEventsFeedPlugin.zip

4. Configure each Store node number 2 or 4 as follow:

   1. cd /app/elasticsearch_nodes/config/MonTier-es-raw-trans-Node-2 
   2. Edit file elasticsearch.yml

5. add to the end of file the parameters:

   Parameter	Values	Description
   montier.events.feed.syslog.protocol	tcp or udp	the protocol used to send syslog
   montier.events.feed.syslog.host	ip addressv4 or valid hostname	target syslog agent hostname
   montier.events.feed.syslog.port	integer 1-65535	target syslog agent port
   montier.events.feed.syslog.ssl	false or true	enable / disable ssl
   montier.events.feed.appname	text A-Z|a-z|0-9 name - 8 chars	syslog application name

Configure syslog agent

For each syslog agent that you have in system perform the following

1. cd /app/flume/syslog_agents/conf/MonTier-SyslogAgent-nn/ (replace nn with agent number)
2. edit file flume_syslog.conf
3. Look  in each each agent for rows with pattern : MonTier-SyslogAgent-nn.sinks.syslogElasticSinknnn.batchSize = (replace nnn with sink name 1 to 10!! and nn with agent number !!)
4. For each row found please add below the line 
   MonTier-SyslogAgent-nn.sinks.syslogElasticSinknnn.serializer.enableLogicalTx = false