2019/11/01

Private lives

In the late 1980s and early 90s, the main business of the company I worked for was providing searchable online periodical content, often with article text, first through CompuServe and later through public and academic libraries. In those days we "rolled our own." We started with some fairly basic software that we then changed and revised and upgraded over the years.

For several years, we logged only error messages, both fatal exceptions and non-fatal errors, such as search failures. We could tell when people found no matches for their search, but we didn't know what the failing search expression contained.

We did not go lightly into logging search expressions. By now many are too young to remember how high the expectation of privacy used to be, especially in the United States. In the early days of online activity, it was assumed that people were entitled to the same privacy as they would have had while doing research at a library or talking with someone on the telephone. And privacy meant privacy, not a pledge to take your information but keep it safe and confidential.

We really didn't want to know what people were searching for. It might be all kinds of embarrassing stuff. We thought of what we would be doing as snooping. But we realized that not saving search expressions was hampering efforts to improve our search server. If we knew why searches failed, we should be able to help customers be more successful, and we would know more about what people expected from our products.

You might find this quaint, but we were nervous about snooping. We didn't want customers to know we could see what they searched for.  We told ourselves we would use "snoopage" only for goodness and niceness, meaning to help diagnose a problem or failure.

I don't remember when this kind of detailed information capture became normal. I think it might have been another decade, or nearly so. I know that long before I retired, the logging of detailed search information had become routine, and it was used for metrics, analysis, marketing, and more.

Shortly before I left, the company was at the point of dealing with European Union privacy regulations. I'm not sure if those are closing the barn door after the horse has already gone. Maybe they're more like not letting the horse run quite as wild and crapping all over the place.

One reason the company was not drawn to mine user data was that the services cost money, then and now. We figured that entitled users to some level of privacy.

Most of the services we use now are "free." Free meaning that we don't pay a fee beyond our internet connection charge. The services are anything but free, however, when you look at what we've given in exchange—something that only a half century ago people would have preserved at all cost.

No comments: