Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The System Health dashboard provides an overview of all devices health states based state based on user-defined metrics. 

The dashboard is composed of multiple cards divided into groups(optional) to display the , each representing the health of a device, optionally divided into device groups.

In addition, the dashboard displays the DPOD health state based on Internal Health Alerts.

Assumptions:

  • The existing alerts infrastructure (with additional few more fields) will provide all data and logic to decide if a sample should be alerted and if it considers an error or warning or good.
  • The alerts mechanism will be the only source of current and future metrics. New fields: Is alert used as health metric, warning threshold, damage points
  • Any health metric must be based on a detailed investigation screen

Prerequisites:

Metric:

...

Metrics

  • The health of each device is based on several user-defined metrics. For example, the CPU of the device.
  • A metric is basically a criteria and a set of thresholds that together define whether the state of the device in that aspect is one of: Good / Waring / Error.
  • Metrics are based on the Alerts subsystem of DPOD:
    • The user can define
    whether a metric is
    • which alerts are part of the
    system health by 
    • System Health by selecting the "System Health Metric" option
     
    • You may edit the metric setting from under [Manage → Alert → Setup Alerts →  Edit Alert] 

Device Health Settings:

  • For each device, the user can define whether the device is displayed in the System Health dashboard, Damage Points Threshold, Total Warnings Threshold
  • For each device, the user can set thresholds and damage points per health metric. - TODO add link to device settings

Device Group Settings:

  • For each device, the user can define the device group and the display order -  TODO add link to device groups

System Parameters (DB) - default values for:

    • .
    • Each alert can be used as a simple alert, as a System Health metric, or both. Using an alert both for alerting and as a System Health metric is recommended, since it makes sure the System Health dashboard will precisely reflect the sent alerts.
  • The following System Health metrics are defined by default:
    • Devices CPU Metric
    • Devices Memory Metric
    • Devices Load Metric
    • Devices Fan Metric
    • Devices Temperature Metric
    • Devices Voltage Metric
    • Devices Space Encrypted Metric
    • Devices Space Temp Metric
    • Devices Space Internal Metric
    • System Errors Metric
    • Device Availability Metric - This is an internal metric based on "Device Resources Monitoring" option selected at the device level, that checks whether the device is available or not.

Device Health Calculation

  • The System Health dashboard calculates the health of each device in the past hour.
  • The past hour is divided to 5 parts:
    • Last 5 minutes (may be configured via "System Health Dashboard Sample Time Range (min.)"

...

Device Card description:

  • Health states:

...

Image Removed

...

Image Removed + background color of card is red

...

  • A single card to display the health of a device - includes:
    • Big icon to indicate system health based on the metrics in the last X (System Parameter) minutes
      • If  no metrics found then health is critical - the device is dead and marked as Critical!
      • If  the "Device Availability" metric last sample is Error or all samples are Error within the last X minutes, the device is marked as Critical.
      • If one error or more are within the last X minutes then device health is marked as Error (no matter the time order of system health)
      • If no errors found but some warning exists then
        • If total damage points for all metrics is bigger than the damage points for a device(for a specific time range) to sustain then device health is Error.
        • If total of warning is bigger than number of warning found for a device than device health is Error
        • else the device health is Warning
      • If no Errors or Warnings found then device health is Good
    • Small icons to display past hour health - using the same logic as the current health
  • Clicking a device card should direct to the device health dashboard

...

    • System Parameter)
    • Previous 10 minutes
    • 3 parts of 15 minutes (the rest of the hour)
  • Each part displays a single icon with the health of the device during that period of time:
IconDescriptionLast 5 minutesOther parts

Image Added

Good

No errors or warnings found in metric samples

(same)
Image Added
WarningWarnings found in metric samples(same)
Image Added
Error

Errors found in metric samples OR
Warnings count exceeded threshold (see below) OR
Warnings damage points exceeded threshold (see below)

(same)
Image Added
Unknown-

No metric samples found (e.g. DPOD alert subsystem was down) OR
The device was unavailable during the entire time period

Image Added + red background color

Critical

No metric samples found (e.g. DPOD alert subsystem was down) OR
The device was unavailable in the last "Device Availability Metric" sample

-
  • Each device may have a total warnings threshold which sets the health of that device to Error in case the number of metrics that are at Warning state exceeds that threshold in a specific time period (see Device Health Settings).
  • Each device may also have a warning damage points threshold which sets the health of that device to Error in case the summary of the damage points of all metrics that are at Warning state exceeds that threshold in a specific time period (see Device Health Settings).
    • Each System Health metric may be assigned with damage points, which should reflect the severity of that warning.
  • For each device, the user can set thresholds and damage points per health metric, which override the default thresholds and damage points defined at the System Health metric level (see Device Health Settings).

Devices Display Options

  • For each device, the user can define whether the device is displayed in the System Health dashboard.
  • The user may define device groups:
    • Each device group has a name and a display order of that group
    • Devices are assigned to one or more device groups with a defined display order
    • For example: Production, Non-production

Device Health Dashboard

  • Clicking a device card in the System Health dashboard opens the Device Health dashboard which displays a detailed view of a specific device health.
  • This dashboard displays all metrics that were part of the device health calculation.
  • For each metric, all samples are displayed in a chart, which displays each sample values when hovering.
  • For analyzing a System Health metric, the user may click "Analyze" when hovering over the metric values. This will take the user to the appropriate dashboard for analyzing the values.
    • Each System Health metric may be assigned with a drill-down dashboard.