Newspapers: Open Your Archives

One of these days, a newspaper currently charging a premium for access to its article archives will do something bold: It will open the archives to the public -- free of charge but with keyword-based advertising at the margins.

I predict that the result will pleasantly surprise the bean-counters. There'll be a huge increase in traffic at first, once people realize they can read their local history without paying a fee. Eventually, though not instantly, the revenues will greatly exceed what the paper had been earning under the old system. Meanwhile, the expenses to run it will drop.

And, perhaps most important, the newspaper will have boosted its long-term place in the community. It will be seen, more than ever, as the authoritative place to go for some kinds of news and information -- because it will have become an information bedrock in this too-transient culture.

I've believed this move is necessary for newspapers for a long time. Almost three years ago, in an article for Library Journal called "Yesterday's Headlines," Rich Wiggins looked at the growing "archive digital divide," in which some people put things on the Web in a permanent way and others put them behind walls. He asked me what I thought. I replied, in part, "I have a feeling that the newspaper industry would be better served by opening up the archives and Googling them (and selling related ads based on keywords entered) than charging for individual searches."

I recognize the institutional and financial hurdles that will make it difficult to pull off in many companies today, even if they like the idea in a general sense. But I also believe it's almost inevitable.

Some major newspapers already do it. I would love to know how much the San Francisco Chronicle, which offers its archives at no charge (and loads search results with too many ads), get in revenue compared with the San Jose Mercury News, which charges for older articles.

For me, again, the heart of this issue is more about a newspaper's community role than revenue, even though I believe the right thing from the community standpoint is also the best from a financial one.

In an important recent posting on the PressThink blog, the Guardian's Simon Waldman explained "The Importance of Being Permanent" -- the notion that Web postings should have permanent, accessible Web addreses, or URLs. He wrote, in part:

Permanence means understanding that when you put something on the Web it should be there for ever: ideally in the same place for perpetuity. It means that if I link to it now, someone else can follow that link in two days, two weeks or two years' time. (I'm not going to lay out the business models in this piece, but I'm not excluding the possibility of pay-to-view; it's the position that counts, not the price.)

Waldman pointed out that the mainstream news industry, with a very few exceptions, doesn't see things this way. The consequence of the industry's tendency to push its older journalism behind a wall, where the initial links disappear and people now have to pay for access, is a loss of identity and diminution of authority.

Jay Rosen believes journalists should demand it, since their work is being hidden from most of the world as a result of such policies (he has more on this in a posting today. As he notes, last week's Harvard conference on Blogging, Journalism & Credibility produced a surprising consensus in favor of opening archives. Maybe that's not so unexpected after all, given that almost all of people in the room were professional journalists and bloggers: "content" types. (It's easy for me, as a proud former member of what newspaper publishers call "cost centers," to advise companies to give up a certain source of revenue for a speculative one.)

There are two immediate questions for publishers. First, how much money are they making today from pay-per-view archives? The Poynter Institute's Bill Mitchell said over the weekend (on a mailing list), "The range of news orgs generating significant revenue from archives with current biz models may be more limited than I had assumed."

That's good news. It means that the financial risk of changing the model is small in many, if not most cases.

Second, and related to that, what are the benefits of shifting to an open archive? I'm utterly in Waldman's camp that it would heighten a newspaper's authority, helping keep it at the center -- or not far from there -- of the community's civic agenda.

Every newspaper of any quality has published hundreds of articles that readers can find nowhere else and which bloggers, among others, would surely cite and point to as a vital part of the permanent record of a community. These include investigative pieces, certain features and other stories. If available upon publication at a permanent URL, they quickly rise in search engine rankings, where others will find them later.

I'm convinced that increasingly sophisticated Web advertising, especially keyword-based text ads, will create a revenue stream of some size for such stories. This will be especially the case when they've moved high enough in search-engine rankings to be found without searching the newspaper's site, but that's not crucial.

A locally targeted ad based on a keyword will bring new kinds of advertisers to the newspaper: small businesses that couldn't afford to buy space in the print edition. This is new money. It may not replace what papers are losing to the online competition, but it's worth something.

Many articles won't have that broad appeal. But they will have special meaning to smaller numbers of people who will want to point to them from their personal sites. They will reinforce people's sense that the newspaper is a medium of record in their lives. That's also worth something.

If I was a publisher with a pay-per-view archive, here's what I'd do:

1) Re-publish every article in the archives with a unique URL, outside the pay-wall. It would be helpful if the articles published since the newspaper went online could have the same URLs, but don't worry if that's too expensive; if the stories are important enough, they'll be found and pointed to. It'll just take a little longer.

2) Leave every new article on the Web at the URL it had upon publication. That's easier.

3) Encourage the readers to use the archives, with house advertisements, website notices e-mail to local librarians and other ways to get out the word.

4) Let local bloggers know that you welcome their links, and that you've made the change in part because they need it, too.

5) If a local blogger points to your article, use Trackback or other such technology to point back. (But be careful of link spamming.)

I don't know if there are ways to share revenues with people who point to news articles, or if that might raise some difficult ethical questions. I do know that the archives should be open in a way that encourages cooperation with bloggers and other Web publishers, even when you're competing in other ways.

Someday soon, some paper is going to try this. It'll be a great experiment. I believe it'll be a successful one, too.

Comments

[Bob Rosenberg]

"It will open the archives to the public -- free of charge but with keyword-based advertising at the margins."

Seems to me that this model may be the ONLY way many newspapers can survive. With internet, blogging, etc., the old revenue models may be moving into an inexorable decline.

[Francis Hwang]

I just wrote a reply to this discussion on my own blog. Just because open archives are good for the public as a whole doesn't guarantee that they're good for publishers; my post goes through some specifics about publishing and tries to come up with areas of possible progress. The first point worth noting is that free news and paid archives are essentially different products serving different audiences, and many publishers just might have the luxury of caring if walling off archives hurts public discourse.

[Todd Lekan]

We've been publishing our archives for free for years now. The ad revenue and audience boost are of exponentially greater benefit than the revenue we once received from paid archives. Ancillary benefits include tremendous and instantaneous good-will from our audience as well as extra credibility amongst the cognoscenti who normally laugh at newspapers' attempts to put walls around their content.

[Francis Hwang]

Todd, are you referring to madison.com? I wonder if that sort of newspaper would have different sorts of economies than, say, the New York Times, which aims to be a national newspaper and won't proportionally benefit from locally targeted ads the way that madison.com will. Conversely I would imagine that a site like madison.com would have a significantly worse history with a single-site paid archive system than a site like nytimes.com. Now, of course, there's no reason that this conversation has to focus on the Times, you could make the case that the edges are just as important as the center ... but I think the center definitely has its own sorts of business pressures to deal with.

[Todd Lekan]

Yes, I'm referring to madison.com. I agree that our success with an ad-supported model does not necessarily transfer to a national site like nytimes.com. We have seen similar success in our smaller publications (wiscnews.com) but I can't really speak to how the model translates to national sites. Were it ever applied, I would be very optimistic about the impending results.

[Francis Hwang]

I think that the significant distinction isn't so much national vs. local, as it is broadcast vs. narrowcast. You could have a national zine all about, say, classic cars, and get pretty good returns from selling ads, because then you've got a well-defined market. But the Times is both mainstream in topic and national (or global) in scope, so it might be harder for them to find well-defined markets for highly targeted ads. I'm sure the Times gets lots of clickthroughs from their Google ads, but I suspect that the difference isn't commensurate to how much more it costs the Times to produce content than it costs, say, somebody blogging about PVRs.

Basically, I think the "long tail" is probably good for the little guys and bad for the big guys. And maybe that's okay. Regardless, I wouldn't expect the Times to extrapolate for their results based on the results of anybody significantly smaller than them.

[Richard Silverstein]

I'm distressed to hear that NYT may be contemplating moving to a subscription based model, though if the fee was reasonable then it might not be too onerous esp. if it gave you unlimited access to articles.

In the meantime, if you're looking for specific articles online I suggest google searches using the title as the keyword search surrounded by quotation marks. I often find articles I'm seeking that have been published in local papers which have affiliation agreements with NYT; or I'll find that another website or blogger has uploaded the article to their own server. Almost every NYT article I link to in my blog uses the RSS Userland permanent NYT link so articles at my site are permanently available (or at least they're available until the NYT shuts down this agreement w. Userland). For more on this, see Joe Clark's link above.

[Jozef Imrich]

Article of note recently placed on Romenesko:

Rick Edmonds : Not so long ago -- three or four years -- online operations were a business afterthought at newspapers. Revenues from the sites were tiny, one percent or less of the total... An Online Rescue for Newpapers?

[Andrew Jankowich]

One unusual aspect of the Times' policy is that two sections of the paper's archives that would seem to have the longest shelf life, movie reviews and travel, have extensive free offerings: 10,000 movie reviews here at http://movies.nytimes.com/ref/movies/reviews/index.html and the Destination and Interest Guides that organize travel articles by location and theme at http://www.nytimes.com/pages/travel/.

[Robert C Worstell]

Just recently reading The Magic Cauldron, by Eric Raymond. This classic work predicts the economics of opening up the "source code" will make far more revenue than closing them up.

[lifebalance]

Most leading national newspapers in India (Times of India, The Hindu, etc) have provided searchable archives. As a consumer of media, I know how much of a benefit it is.

[Anna]

"economics of opening up the "source code" will make far more revenue than closing them up"

Or as Doc Searls put it here - "...The Web obeys new structural and economic laws that seem to have more in common with the mathematics of loaves and fishes than with the traditional economics of scarce resources and diminishing returns..."

[Trustpharma]

Google indexing (monthly) Yes/No.
If Yes then what is the specify date to crawl the website.

Welcome to www.trustpharma.com, your comprehensive and safe online pharmacy for obtaining generic medications. Our online medical store supplies only top-quality generic drugs that are exact equivalent of the brand name drugs.

[bill cosby]

Low cost high quality Manufactures of Hydraulic and stainless steel hose assemblies

[Leon]

newspaper open your archives!

[bill cosby]

specialized in worldwide distribution of electronic components with emphasis on obsolete and hard to find devices

[georges]

inevitable, yes it is

[Manon Manon]

We are computer Hardware and Software traders. We are specialized in Computer AMC, Professional Project.

[Springer]

Newspapers need to open up their content to te web!
I think alike Dan Gillmor.