Pop Tarts and Beer. Who'd have thunk it?

Wal-mart would. Even if you don't work in the marketing field, you may find this NYT piece interesting. It talks about the practice by Wal-Mart (and nearly every other major retail operation) of collecting, storing, and analyzing sales data at their stores. In one example, they talk about the series of hurricanes that hit Florida this year, and how their sales data led them to predict things that would be hot sellers as people prepared for the arrival of the storms.

The experts mined the data and found that the stores would indeed need certain products - and not just the usual flashlights. "We didn't know in the past that strawberry Pop-Tarts increase in sales, like seven times their normal sales rate, ahead of a hurricane," Ms. Dillman said in a recent interview. "And the pre-hurricane top-selling item was beer."

Thanks to those insights, trucks filled with toaster pastries and six-packs were soon speeding down Interstate 95 toward Wal-Marts in the path of Frances. Most of the products that were stocked for the storm sold quickly, the company said.

What they are doing here has been around for a while. It's called Data Mining, and people selling retail sales computer solutions (including yours truly) have been heavily invested in it for a number of years. The basic concept is that stores can seek to maximize sales by product selection and positioning in the store based on past buyer patterns. You may be surprised to learn that they not only track what you buy, but also what time of day you buy it, your gender, and a host of other facts.

For example, it will likely not shock you to find out that a woman with a small child in tow, pushing a full cart in a grocery store on a weekday morning, who purchases a gallon of milk might also buy eggs and sliced cheese. That's just a no brainer, and the products are already generally positioned near each other. But the people who have made their living and their reputations in data mining have found seemingly incongruous patterns which turn out to be valid buyer trends and result in higher profits.

One of the earliest and most quoted examples is a case in the early 90's concerning diapers and beer. On the surface, these may look like two products that don't have much in common. In fact, daytime sales in large grocery stores showed no relationship in the sales of the two products. There was, however, a data mining study done by a chain of convenience stores operating across the South that turned up an odd anomaly. Evening purchases by men rang up sales of both diapers and beer on the same purchase at a much higher rate than could be explained by random chance. With no explanation offering itself, the company acted on that data and ordered all of their outlets to move their diaper displays over next to the beer coolers in their stores. Over the next three months, the stores' beer sales in the evening shift nearly tripled and the guy who discovered the pattern was proclaimed a genius.

It turned out that daytime, grocery store sales of diapers were made by mothers who didn't buy beer at a rate higher than any other demographic group. In the evenings, however, the main purchasers of diapers were the fathers who were sent out on late night errands when the house was running short of baby supplies. Since they were already out at the store anyway, the dads were far more likely to grab a six pack. Finding that connection allowed the stores to realize a huge surge in sales and profits.

There are a lot of other examples, but it might give you pause next time you're out shopping. Every purchase you make is scanned into a massive data collection system to be analyzed to the n'th degree. Happy shopping! (And grab me a case of Guinness while you're out.)