A friend of mine asked me how he could remove invalid data points from the measurements made by the ultrasonic range finder in his Arduino project. A quick search got me to this page, but some of the links were dead and there was no complete example on the page anyway, so I decided to roll my own.
Here’s my take, written in Python since I didn’t have an Arduino handy for testing. This is based on dropping outliers from recent measurements to come up with a “fixed average”:
#!/usr/bin/env python
# Example on filtering out invalid data points in a stream, coming in from for
# example an ultrasonic distance sensor.
# This test data has two invalid data points; 310 and 210
data = [40, 41, 40, 39, 42, 45, 310, 43, 41, 44, 38, 120, 121, 117, 120, 120, 210, 110, 112, 114, 117, 113, 78, 80, 78, 74, 60, 70, 76]
raw_values = [];
while data :
# Read a raw value
raw_value = data.pop()
# Keep last 5 values stored in an array
raw_values.append(raw_value)
if len(raw_values) > 5:
raw_values.reverse()
raw_values.pop()
raw_values.reverse()
# For debugging only; Print out current stored raw values
for raw_value in raw_values:
print raw_value,
# Calculate the raw average of all the stored values; This value will be
# thrown off by invalid values.
avg = round(sum(raw_values) / len(raw_values))
print ":",
print avg,
# Drop the highest and lowest stored value if we have enough values.
sorted = list(raw_values);
if len(raw_values) >= 3:
sorted.sort()
sorted.pop()
sorted.reverse()
sorted.pop()
# And calculate an average based on the remaining three values. This is
# closer to correct, but will be thrown off if there is more than one
# invalid value currently stored.
fixed_avg = round(sum(sorted) / len(sorted));
print ":",
print fixed_avg,
# Print the difference between the raw average and the fixed average:
print "(",
print abs(avg - fixed_avg),
print ")"
