Newspapers Should Open Archives

On another blog in early 2005 I posted an essay entitled, “Newspapers: Open Your Archives” — a plea to the newspaper industry to recognize the value it had hidden away behind pay-walls. I’m republishing it below, in part because I think the logic has held up even as newspapers have continued to hold back.

One of these days, a newspaper currently charging a premium for access to its article archives will do something bold: It will open the archives to the public – free of charge but with keyword-based advertising at the margins.

I predict that the result will pleasantly surprise the bean-counters. There’ll be a huge increase in traffic at first, once people realize they can read their local history without paying a fee. Eventually, though not instantly, the revenues will greatly exceed what the paper had been earning under the old system. Meanwhile, the expenses to run it will drop.

And, perhaps most important, the newspaper will have boosted its long-term place in the community. It will be seen, more than ever, as the authoritative place to go for some kinds of news and information – because it will have become an information bedrock in this too-transient culture.

I’ve believed this move is necessary for newspapers for a long time. Almost three years ago, in an article for Library Journal called “Yesterday’s Headlines,” Rich Wiggins looked at the growing “archive digital divide,” in which some people put things on the Web in a permanent way and others put them behind walls. He asked me what I thought. I replied, in part, “I have a feeling that the newspaper industry would be better served by opening up the archives and Googling them (and selling related ads based on keywords entered) than charging for individual searches.”

I recognize the institutional and financial hurdles that will make it difficult to pull off in many companies today, even if they like the idea in a general sense. But I also believe it’s almost inevitable.

Some major newspapers already do it. I would love to know how much the San Francisco Chronicle, which offers its archives at no charge (and loads search results with too many ads), get in revenue compared with the San Jose Mercury News, which charges for older articles.

For me, again, the heart of this issue is more about a newspaper’s community role than revenue, even though I believe the right thing from the community standpoint is also the best from a financial one.

In an important recent posting on the PressThink blog, the Guardian’s Simon Waldman explained “The Importance of Being Permanent” – the notion that Web postings should have permanent, accessible Web addreses, or URLs. He wrote, in part:

Permanence means understanding that when you put something on the Web it should be there for ever: ideally in the same place for perpetuity. It means that if I link to it now, someone else can follow that link in two days, two weeks or two years’ time. (I’m not going to lay out the business models in this piece, but I’m not excluding the possibility of pay-to-view; it’s the position that counts, not the price.)

Waldman pointed out that the mainstream news industry, with a very few exceptions, doesn’t see things this way. The consequence of the industry’s tendency to push its older journalism behind a wall, where the initial links disappear and people now have to pay for access, is a loss of identity and diminution of authority.

Jay Rosen believes journalists should demand it, since their work is being hidden from most of the world as a result of such policies (he has more on this in a posting today. As he notes, last week’s Harvard conference on Blogging, Journalism & Credibility produced a surprising consensus in favor of opening archives. Maybe that’s not so unexpected after all, given that almost all of people in the room were professional journalists and bloggers: “content” types. (It’s easy for me, as a proud former member of what newspaper publishers call “cost centers,” to advise companies to give up a certain source of revenue for a speculative one.)

There are two immediate questions for publishers. First, how much money are they making today from pay-per-view archives? The Poynter Institute’s Bill Mitchell said over the weekend (on a mailing list), “The range of news orgs generating significant revenue from archives with current biz models may be more limited than I had assumed.”

That’s good news. It means that the financial risk of changing the model is small in many, if not most cases.

Second, and related to that, what are the benefits of shifting to an open archive? I’m utterly in Waldman’s camp that it would heighten a newspaper’s authority, helping keep it at the center – or not far from there – of the community’s civic agenda.

Every newspaper of any quality has published hundreds of articles that readers can find nowhere else and which bloggers, among others, would surely cite and point to as a vital part of the permanent record of a community. These include investigative pieces, certain features and other stories. If available upon publication at a permanent URL, they quickly rise in search engine rankings, where others will find them later.

I’m convinced that increasingly sophisticated Web advertising, especially keyword-based text ads, will create a revenue stream of some size for such stories. This will be especially the case when they’ve moved high enough in search-engine rankings to be found without searching the newspaper’s site, but that’s not crucial.

A locally targeted ad based on a keyword will bring new kinds of advertisers to the newspaper: small businesses that couldn’t afford to buy space in the print edition. This is new money. It may not replace what papers are losing to the online competition, but it’s worth something.

Many articles won’t have that broad appeal. But they will have special meaning to smaller numbers of people who will want to point to them from their personal sites. They will reinforce people’s sense that the newspaper is a medium of record in their lives. That’s also worth something.

If I was a publisher with a pay-per-view archive, here’s what I’d do:

1) Re-publish every article in the archives with a unique URL, outside the pay-wall. It would be helpful if the articles published since the newspaper went online could have the same URLs, but don’t worry if that’s too expensive; if the stories are important enough, they’ll be found and pointed to. It’ll just take a little longer.

2) Leave every new article on the Web at the URL it had upon publication. That’s easier.

3) Encourage the readers to use the archives, with house advertisements, website notices e-mail to local librarians and other ways to get out the word.

4) Let local bloggers know that you welcome their links, and that you’ve made the change in part because they need it, too.

5) If a local blogger points to your article, use Trackback or other such technology to point back. (But be careful of link spamming.)

I don’t know if there are ways to share revenues with people who point to news articles, or if that might raise some difficult ethical questions. I do know that the archives should be open in a way that encourages cooperation with bloggers and other Web publishers, even when you’re competing in other ways.

Someday soon, some paper is going to try this. It’ll be a great experiment. I believe it’ll be a successful one, too.