Symptoms
Some transactions are not showing WSM Payloads, even though WSM subscription was requested for them.
In /logs/wsmAgents/agent-flume_MonTier-WsmAgent-X.log (where X is the number of the DPOD WSM agent you use), you will see the following message(s):
12/07/2016 02:12:29,852- WARN o.m.a.f.h.NHttpFileServer [I/O-dispatch-3] wsmHttpSource1 -98b515b9-ed96-45dd-a999-248f6f19f1a3- Request ignored - With size 18615431 that limit 10485760
Overview
Each WSM event contains 4 payloads: Front-End request, Back-End request, Back-End response and Front-End response, the size of a single event equals to the size of all 4 payloads combined.
The DataPower batches several transactions (WSM events) together before sending them to DPOD as a single request- The default is grouping 20 events together.
DPOD's WSM Agent sets a size limit for incoming requests, the default is 10mb, meaning, DPOD will drop any requests that are larger than 10mb
Assuming you have a transaction with payload size of 1mb (300kb back-end and front-end requests, and 200kb back-end and front-end responses), and the DataPower grouped 20 of them before sending them to DPOD, the total request size will be 20 x 1mb = 20mb , so DPOD will drop this request and will not show the payloads.
Changing how many payloads the DataPower will batch together
You can effect the number of transactions the DataPower will send to DPOD in each batch and the maximum batch size - all the configuration changes are done in DPOD and not in the DataPower.
You will need to stop DPOD UI agent from the CLI before changing the following values, and start it again afterwards.
edit /app/ui/MonTier-UI/conf/MonTierUI.conf
the relevant configuration keys are:
wdp.wsm.push.max_elements - how many transactions the DataPower will group together (the default is 20)
wdp.wsm.push.max_envelope_size -max batch size that the DataPower will send to DPOD (the default is 10485760 = 10mb)
Changing the max batch size DPOD can handle
You will need to stop DPOD WSM agent from the CLI before changing the following values, and start it again afterwards.
edit /app/flume/wsm_agents/conf/MonTier-WsmAgent-X/flume_wsm.conf (where X is the number of the WSM Agent you want to effect)
The relevant configuration key is:
MonTier-WsmAgent-X.sources.wsmHttpSource1.requestSizeLimitInBytes - max batch size that DPOD will process (the default is 10485760 = 10mb)
Best Practices
It is much better to lower the number of the batched payloads (wdp.wsm.push.max_elements) than to increase the batch size DPOD can accept (wdp.wsm.push.max_envelope_size and wsmHttpSource1.requestSizeLimitInBytes) - making more smaller requests will allow both the DataPower and DPOD to work in parallel and will require less memory on both appliances. specifically, setting wsmHttpSource1.requestSizeLimitInBytes to a high value may put some stress on DataPower and DPOD machines with a low amount of RAM.
If you still need to change the batch size because a specific transaction/batch of transaction is larger than 10mb, the best thing will be to raise the limit only for one specific WSM agent and choosing this agent when subscribing to WSM.
