IBM DataPower Operations Dashboard v1.0.10.0

A newer version of this product documentation is available.

You are viewing an older version. View latest at IBM DPOD Documentation.

Increase Storage for Index Sets


This procedure should be performed only after consulting with product support team


Depending on your data retention configuration, you may find that you need to increase the amount of data DPOD is able to store. In those circumstances, you will need to perform the steps below.

  1. Ensure the disks mounted to the /data path have enough free space to accommodate your needs.
    The maximum increase amount (MAX_INC_IX_SIZE) is calculated as the total free space of /data - 15GB. The information regarding /data storage can be found on the OS screen.
  2. To calculate the correct Index set increase, consult these parameters (available on the Store screen)
    1. The number of days currently stored: Index Sets table (this is Newest Document Time - Oldest Document Time)
    2. The amount of storage you are using: Index Sets table (size)
    3. the number of indexes allocated: Index Sets table (Indices count)
  3. As a general rule of thumb, increase the Index Sets size together with their number. Ideally each index will be large enough to contain information for 1.5-2 days. MAX_INC_IX_NUMBER
  4. Additionally, increase memory by 32-96GB for each additional 1TB Data stored. The exact value depends on data variance and acceptable response time from the queries.
  5. Once you have calculated values for MAX_INC_IX_NUMBER and MAX_INC_IX_SIZE , go to file /app/hk_retention/MonTier-HK-ESRetention/conf/MonTierHousekeeping.conf
  6. edit the file. Change the number of indexes with MAX_INC_IX_NUMBER and the total storage allocated with MAX_INC_IX_SIZE. 
  7. for example, to extend the wdp-syslog - change the following properties:

# WDP Syslog tran index set number of indices, usually 20
elasticsearch.index_set.wdp-syslog-tran.indices_count=<MAX_INC_IX_NUMBER> 

# WDP Syslog tran index set max size - NOTE the G for GB at the end.
elasticsearch.index_set.wdp-syslog-tran.max_size=<MAX_INC_IX_SIZE>G


IBM DataPower Operations Dashboard (DPOD) v1.0.10.0