Missing Data? Nah, Never Happened!

This post concerns how missing precipitation data due to equipment outages is treated by the National Weather Service (NWS).

To begin, lets start with today’s ‘climate data’ issued by the NWS for Lawrence and in particular, the rainfall data:

130 AM CDT WED AUG 16 2017



                VALUE   VALUE  FROM      YEAR
  MAXIMUM         88     90     -2       82
  MINIMUM         65     66     -1       64
  AVERAGE         77     78     -1       73

  YESTERDAY        0.00   0.16  -0.16     0.00
  MONTH TO DATE    0.02   1.75  -1.73     1.37
  SINCE JUN 1      9.32  11.39  -2.07     7.32
  SINCE JAN 1     22.34  25.28  -2.94    21.96

Wait a minute!

Didn’t we have a big rainfall event on August 5 that dropped 2 to 3 inches of rain in the Lawrence area – my so-called ‘Encore Storm’?   I measured 2.32 inches on the 5th of August.  The 7-station CoCoRaHS average for the Lawrence stations was 2.39 inches.  The radar precipitation estimate indicated even more rainfall up toward the airport, as did neighboring CoCoRaHS data around the airport (see charts below).


NWS radar estimated precipitation for August 5 and 6.  The two-inch line is located east-west across Lawrence with greater amounts to the north.  This radar estimate, produced by an algorithm that adds up real-time radar sweeps, is slightly underestimating the actual amounts.


CoCoRaHS station precipitation amounts with an estimate for the Lawrence Airport precipitation amount by interpolating between the two nearest CoCoRaHS observations.  The estimate for the airport (rounding down) is 3.01 inches!  This is consistent with the radar image showing greater precipitation up toward the airport than in most of Lawrence.

Clearly something is amiss!  Well, lets look at another NWS ‘climate’ product issued daily:


                                          STATION:   Lawrence KS
                                          MONTH:     AUGUST
                                          YEAR:      2017
                                          LATITUDE:   39  0 N
                                          LONGITUDE:  95 13 W

1   2   3   4   5  6A  6B    7    8   9   10  11  12  13   14  15   16   17  18
                                     12Z  AVG MX 2MIN

 1  86  59  73  -6   0   8 0.00    M    M  2.9  7 320   M    M   1 12      9 340
 2  89  60  75  -4   0  10 0.00    M    M  2.3  8 280   M    M   0 1      11 270
 3  84  63  74  -5   0   9 0.00    M    M  7.1 18 290   M    M   1 1      24 280
 4  80  51  66 -13   0   1 0.00    M    M  3.7 10 340   M    M   1        14 330
 5  69  55  62 -17   3   0    M    M    M  8.8 21  50   M    M   8 1      28  60
 6  77  65  71  -8   0   6    T    M    M  4.5 10 330   M    M   9        14 330
 7  80  62  71  -8   0   6 0.00    M    M  3.9 12  40   M    M   2 1      16  10
 8  82  58  70  -9   0   5 0.00    M    M  3.3 12  80   M    M   1 1      15 100
 9  84  58  71  -7   0   6 0.00    M    M  4.0 14 200   M    M   2 1      16 200
10  87  64  76  -2   0  11    T    M    M  1.8 10 150   M    M   3 12     13 150
11  84  62  73  -5   0   8 0.02    M    M  5.5 14 320   M    M   4        17 320
12  81  59  70  -8   0   5 0.00    M    M  4.2 10 120   M    M   1        14  80
13  82  58  70  -8   0   5 0.00    M    M  6.1 13 130   M    M   4        15 150
14  86  66  76  -2   0  11 0.00    M    M  6.3 13 140   M    M   5        16 130
15  88  65  77  -1   0  12 0.00    M    M  5.9 14 130   M    M   2 1      19 150
SM 1239  905         3 103  0.02    M     70.3          M       44
AV 82.6 60.3                               4.7 FASTST   M    M   3    MAX(MPH)
                                 MISC ---->  # 21  50               # 28   60

This is the so-called F-6 data form.  Again we see the total so far for the month of 0.02 inches, which occurred at the airport on August 11th.  However, note that on August 5, in the precipitation column, is an ‘M’.  This means the data was ‘missing’ that day, or unavailable due to an outage of the rainfall measuring equipment at the Lawrence Airport.   So the total for the month in Lawrence remains 0.02 inches on both of these NWS issued and disseminated products. And, this is despite the conflicting data on the NWS radar image or the CoCoRaHS data, which is ‘affiliated’ with the NWS.   And this exact same data is what the Lawrence Journal World and other media outlets publish as well.  Out of sight (missing), out of mind, one supposes!

This has happened a number of times this year, as it usually does every year.  Over time, the cumulative error mounts.

For example, as of 1 AM on August 16, the NWS reports 0.02 for the month of August at Lawrence while the 7-station CoCoRaHS average for the month of August in Lawrence is 3.23 inches.

In July there were no outages.  The airport reported 3.18 inches while the 7-station CoCoRaHS average was 3.15 inches.  Very consistent – that month.  Most excellent!  (Airport + 0.03 inches, or 101 percent of the 7-station Lawrence average.)

In June no outages were seen, but the airport reported 6.29 inches versus the 7-station CoCoRaHS average of 8.59 inches. (Airport is minus 2.3 inches, or 73 percent of the 7-station Lawrence average.)  Why the discrepancy?  Probably because sometimes the gauge, in addition to going out completely, also reads too low but still produces data – another form of missing data.  But instead of ‘missing’ the day entirely, the gauge sends out bogus low-balling data.  This is also a common occurrence with this type of automated gauge!

In May, the airport came in with 2.89 inches versus the 7-station Lawrence average of 3.36 inches. (Airport minus 0.48 inches, or 86 percent of the 7-station Lawrence average.)  May 31 was missing at the airport when the average for the day in Lawrence was 0.19 inches.

In April the airport came in with 6.89 inches compared with the 7-station Lawrence average of 7.57 inches. (Airport minus 0.68 inches, or 91 percent of the 7-station Lawrence average.)

In March the airport came in with 2.37 inches compared to the 7-station Lawrence average of 2.95 inches. (Airport minus 0.42 inches, or 80 percent of the 7-station Lawrence average.)

January and February combined came with with 1.01 inches for the airport versus 1.38 inches for the Lawrence average. (Airport minus 0.37 inches, or 73 percent of the 7-station Lawrence average.)

Except for August these are not huge monthly differences, but for the entire year they add up.

As of August 16, at 1 AM, the Lawrence airport had an NWS reported total precipitation for the year to date of 22.34 inches, or 2.94 inches below the average to date of 25.28 inches.

The 7-station Lawrence CoCoRaHS year to date total was 29.83 inches, or 4.55 inches above the average of 25.28 inches – a rather different rainfall story!

The year to date difference between the Airport and the 7-station Lawrence average is now 7.49 inches, which puts the Airport cumulative precipitation for 2017 to date of 75 percent of the 7-station Lawrence average.  (Of course, the approximate miss of three inches at the airport for the August 5-6 event is a large portion – 40 percent – of this deficit.)

My own CoCoRaHS station, located in southwest Lawrence near Inverness and Clinton Parkway, was up to 31.03 inches for the year to date, or 5.75 inches above the average for Lawrence to date and 8.69 inches above the airport value – an even larger discrepancy. However, I am located about 6 1/2 miles southwest of the airport.

The point is, bad numbers add up!  Ultimately, they can tell a quite inaccurate story!

Why does the NWS knowingly put out such poor quality precipitation data – and consistently so over time?  In a word, my guess would be ‘bureaucracy’ – they have other priorities.  Another possibility is that the leadership of the agency just doesn’t care.  Or, one could probably say, in more or less equal measure, BOTH!

That question, I suppose, is kind of like asking why motorists drive stupidly, over and over again and all the time?  Is it just rudeness, or sheer stupidity?  How does one discern the difference?  Well, it is probably both rudeness and stupidity.  How much of each, who knows.  However, the end result is the same either way – a dangerous driving environment!

Analogously, the same might apply here.  Whether knowingly sending out such poor quality precipitation data after the fact over periods of months and years is because of the sheer weight of a large, slow-moving bureaucracy that has a broader mandate and higher priorities, or leadership in the agency that just doesn’t care about the quality of the precipitation data that they disseminate, or both, the result is the same either way:  Poor quality, low-balling precipitation data being disseminated to the public and  retransmitted by the media.

In conclusion, take reported NWS precipitation data – at least that reported at non-major airports –  with a grain of salt.  (Major airports, like KCI (MCI), and co-located NWS sites, like TOP, use different gauges and do not suffer from such large and consistent errors, but that is a topic for a later post.)  For a much superior alternative at non-major airport climate data sites, use CoCoRaHS data, or your own rain gauge!

CoCoRaHS, as well, comes up with averages for each station that has reported for at least one entire water year using algorithms they have developed and these are probably better data averages, too.  This is a small group that cares about what they do and devotes a lot of time and energy in their efforts to provide precipitation data as consistently and accurately as possible.  Of course, no precipitation data measuring system is perfect, but I would venture to say that the CoCoRaHS data is about as good as can be achieved.

Perhaps one of these days, the NWS will get up to speed and correct this little (or major) data problem they have had for decades now, and not just in Lawrence but at smaller airport sites throughout the country.  Locally, this also includes the downtown airport in Kansas City (MKC), which has had the same kind of low-balling precipitation errors for years.

All that said, the NWS as a whole does a pretty darned good job these days of fulfilling their primary mandate of daily weather forecasting, providing storm warnings, flood warnings, scientific reasoning and the myriad other less visible services they provide to the public (and to TV weather broadcasters so they can pretend they are coming up with the forecasts themselves!).  Given this, it does make one wonder – why does the agency provide such poor-quality information in this one important dataset.   Yet, their own products clearly tell the story, as I have used Lawrence as an example to document. And this story is not just a local phenomenon, but is repeated frequently nationwide – and I would venture to say probably at virtually every non-major airport that reports exclusively with this type of automated gauge.

On a later post, I will touch on the reasons why the automated rain gauges go down or often low-ball precipitation amounts at the non-major airports that also serve as official climate stations.

Until then, I see Lawrence is located along the northwest edge of a severe thunderstorm watch and storms are bubbling up from Wichita to Kansas City.  Unless they back build, it appears the storms will miss the Lawrence area, though.

Time for a closer look..












Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s