ampli.fi blog

A friend of mine asked me how he could remove invalid data points from the measurements made by the ultrasonic range finder in his Arduino project. A quick search got me to this page, but some of the links were dead and there was no complete example on the page anyway, so I decided to roll my own.

Here’s my take, written in Python since I didn’t have an Arduino handy for testing. This is based on dropping outliers from recent measurements to come up with a “fixed average”:

#!/usr/bin/env python

# Example on filtering out invalid data points in a stream, coming in from for
# example an ultrasonic distance sensor.

# This test data has two invalid data points; 310 and 210
data = [40, 41, 40, 39, 42, 45, 310, 43, 41, 44, 38, 120, 121, 117, 120, 120, 210, 110, 112, 114, 117, 113, 78, 80, 78, 74, 60, 70, 76]
raw_values = [];

while data :
  # Read a raw value
  raw_value = data.pop()

  # Keep last 5 values stored in an array
  raw_values.append(raw_value)
  if len(raw_values) > 5:
    raw_values.reverse()
    raw_values.pop()
    raw_values.reverse()
  
  # For debugging only; Print out current stored raw values
  for raw_value in raw_values:
    print raw_value,
  
  # Calculate the raw average of all the stored values; This value will be
  # thrown off by invalid values.
  avg = round(sum(raw_values) / len(raw_values))
  print ":",
  print avg,

  # Drop the highest and lowest stored value if we have enough values.
  sorted = list(raw_values);
  if len(raw_values) >= 3:
    sorted.sort()
    sorted.pop()
    sorted.reverse()
    sorted.pop()

  # And calculate an average based on the remaining three values. This is
  # closer to correct, but will be thrown off if there is more than one
  # invalid value currently stored.
  fixed_avg = round(sum(sorted) / len(sorted));

  print ":",
  print fixed_avg,
  
  # Print the difference between the raw average and the fixed average:
  print "(",
  print abs(avg - fixed_avg),
  print ")"