Standard Deviation Threshold
Standard Deviation in Emplifi Agent reports and utilities is useful for focusing on data that is beyond what is "typical". In basic terms, standard deviation is a statistical concept that shows how close or far away from the average of a set of data the entire set of data is.
Emplifi Agent calculates the standard deviation for a set of data and compares the value of the final period in the Frequency report to that calculated threshold value. If the value exceeds the threshold, it is deemed to be a statistically significant anomaly, and shows up in the report so that someone reviewing the report can further analyze the root cause. Utilizing the standard deviation allows brands to focus their time on statistically significant issues instead of having to look at frequency data for all products.
Below is an example with two products. The actual math involved is explained following the example.
Example
Both Product A and Product B averaged 4.75 complaints per month over the 12-month period shown in the following table. But while the number of complaints each month for Product A was fairly consistent, Product B had very inconsistent performance.
The Standard Deviation value quantifies this fact. It tells us that Product A typically has no more than one (.87) complaint more than the average. On the other hand, it is typical for Product B to have almost three (2.83) more complaints than average.
Month | Product A | Product B |
|---|---|---|
January | 5 | 1 |
February | 4 | 6 |
March | 4 | 3 |
April | 4 | 9 |
May | 4 | 2 |
June | 5 | 5 |
July | 5 | 2 |
August | 5 | 8 |
September | 5 | 7 |
October | 4 | 1 |
November | 5 | 6 |
December | 7 | 7 |
Totals | 57 | 57 |
Average | 4.75 | 4.75 |
Std Dev | 0.87 | 2.83 |
Calculating Averages
The average for a column is found by taking the column total and dividing by the number of entries in the column. For both columns, the total was 57, and there were 12 entries. So the average is 57 / 12 = 4.75.
Calculating Standard Deviation
To calculate the standard deviation of this data, follow these steps:
Subtract the average from each piece of data, and then square that value. For example, for January:
Product A: 5 - 4.75 = .25.... 0.252 = 0.0625
Product B: 1 - 4.75 = -3.75.... -3.752 = 14.0625
Add up all the squared values in each column and divide by the number of rows minus 1, in this case 11.
Product A: the squared values add up to 8.25.... 8.25 / 11 = 0.75
Product B: the squared values add up to 88.25.... 88.25 / 11 = 8.0227
The final step is to take the square root of this last number
Product A: SQRT(0.75) = 0.87
Product B: SQRT (8.0227) = 2.83.
These numbers are the standard deviation -- how far from the average the data can generally be found.
Setting the Standard Deviation Threshold in Emplifi Agent compares the last period's data to the entire data set. The Threshold Value is calculated as follows:
Threshold Value = (Standard Deviation * Standard Deviation Factor) + Average
The report will only display rows of data where the final period’s value is greater than the Threshold Value. For the example above, the Threshold Value is (0.87 * 1.00) + 4.75, or 5.62. The number of complaints in December for Product A (7) is above this value, so it would be displayed.