My most recent article is now available online:

Libraries, openness, and the global community
Panlibus Newsletter, Issue 12, Pg. 11, April 2009

In it I talk about ‡biblios.net and openness in libraries.

The purpose of this blog was set out back in March of ‘07 with an introductory post titled Ali Baba and the Forty Thieves. In it, I wrote about how we’d try to make the case for open source, and help distinguish between open-source and proprietary development/business models so that libraries could understand why open source is so important. Open source was still a somewhat revolutionary concept at the time and defending its viability has been a key focus not only of this blog but of LibLime itself.

While I have always assumed that that open source would become a de facto method for producing software in libraries, I couldn’t have anticipated how quickly it would happen. For instance, Library Journal’s most recent Automation Systems Marketplace reported that of the 607 ILS contracts signed in 2007, 57 of those, or roughly 9.4%, were LibLime customers implementing open-source ILS solutions with Koha. Marshall Breeding, referring to his report, acknowledged recently that “this figure may represent the beginnings of a major trend (Library Technology Reports; Chicago, Nov/Dec 2008; Vol.44, Iss.8, p.19).” Last year (2008) saw even larger growth in the LibLime customer base, not to mention the growth other companies now supporting open source have seen–the upcoming Automation Systems Report covering 2008 is sure to confirm this trend.

There are many reasons behind this upsurge of interest in open source, but I believe the most fundamental reason is philosophical. Libraries have a mission to facilitate open access to ideas and information for their communities, and that mission aligns itself perfectly with the open-source movement. Libraries are about openness, and open source is about openness.

However, while this blog, LibLime, and the industry as a whole, have been focusing primarily on making the case for open source in libraries, it’s time for our collective dialogue about the topic of openness to expand. Open source has become somewhat mainstream and LibLime needs to take our leadership position beyond just defending and promoting open source–there are other types of open that are critical to our industry as well.

The Elevator Pitch

Back in November of last year, during a management planning session, Galen Charlton, LibLime’s VP of Research and Development, challenged me to present LibLime’s ‘Elevator Pitch’. In other words, what’s the pitch for LibLime that can be delivered in the time span of an elevator ride? As it turns out, it’s a very useful exercise, and one that I hadn’t done since we created our first vision statement back in 2005. The company vision and philosophy remain fundamentally the same, but our scope has definitely expanded and it was time time for a new ‘Elevator Pitch’. So here goes:

Libraries have a mission to facilitate open access to ideas and information for their communities; LibLime supports libraries in their mission by providing commercial services on open platform solutions in four key areas:

1. LibLime creates and supports library-relevant Open-Source software, such as Koha ILS, Masterkey Metasearch and YakPac;
2. LibLime promotes and facilitates use of Open Standards with the OpenTranslators Web Service, providing library developers with standards-based access to over 9,000 subscription databases;
3. LibLime provides a free platform for Open Data via ‡biblios.net, including a metadata productivity suite and the world’s largest database of freely-licensed library records;
4. LibLime helps libraries form Open Communities of accessible digital collections, virtual exhibitions, and cultural preservation & celebration, with Kete Community Digital Archive solutions.

Each of these areas of focus represent types of open that are all critical to the library industry. We’ve spent the last four years explaining the benefits of open source, so in this post, I’ll introduce and explore one of these other types of open–Open Data–and make the case for why it’s so important.

Open Data

Historically, libraries haven’t had openly-licensed, community-maintained sources of library metadata. What we do have are:

• National libraries, and the Library of Congress, that create records and make them available freely via Z39.50 or on CD-ROM. These types of databases are openly-licensed in the sense that they are often public domain and free. However, the databases themselves are tightly managed in a top-down fashion, and often don’t contain metadata for many of the materials libraries own.
• Membership-driven organizations, consortia, and ILS vendors often provide subscription-based access to their comparatively large metadata databases, and let members add and maintain the database. This solves the community-maintained piece, but the underlying data is typically not available to non-members and is viewed as the intellectual property of the organization hosting access to the platform, rather than collectively owned by the users.

The philosophy behind an Open Data movement scarcely needs an explanation to a library audience. The mission of most libraries is to provide open, free access to ideas and information. Certainly that same mission applies to the metadata created BY libraries. If we can’t freely share the stuff we’re creating among ourselves, how effective can we possibly be at sharing with our communities?

And yet, historically, there’s been no such thing as openly-licensed, community maintained source of library metadata. So what’s the tipping point for Open Data? There have traditionally been three barriers:

• Lack of available data;
• Lack of appropriate licensing;
• Lack of supporting technology.

However, over the past few years, the pieces have started to fall into place, and we’re now seeing the beginnings of an Open Data movement.

Large amounts of public domain data are now available

2007 saw the launch of the Open Library project, with a goal of creating ‘a page for every book’. Libraries responded by donating over 30 million of their MARC Bibliographic Records, making them freely available by uploading them to the Internet Archive, thereby placing them forever into the public domain. Notable additions to the effort were data sets obtained from the Library of Congress (over 7 million records) as well as UK ILS vendor Talis (over 5 million records). Around the same time, The Library of Congress Authority file surfaced publicly in MARCXML format. And LoC itself opened up access to their records via an XML web service (MARCXML) using LCCN as an identifier, making it possible to access newly created and modified records records more easily.

The First Open Data License

Also in 2007, Talis spearheaded creation of the Open Data Commons Public Domain Dedication and License (PDDL). This was a significant development as PDDL is the first license to address data, and in particular, sets of data as part of a database. As such it’s suitable for licensing both large sets and individual records of library bibliographic, authority, and other library metadata.

The PDDL does for data what licenses like the GPL do for software. PDDL ensures that anyone can use, modify and distribute the records, or the entire database itself.

The Web really took off as an application platform

Momentum has been building in the Web world around the Web as an application program. Long gone are the days when the Web consisted of a bunch of static, clunky HTML pages. Full-featured desktop-style applications like Gmail and Google Maps surfaced, and some of the larger organizations and communities saw benefit in releasing the toolkits they used to create these applications as open-source tools for all developers to use. Yahoo released the YUI toolkit, and other JavaScript toolkits from the open source community followed like ExtJS. Google released it’s Gears project, a powerful tool permitting offline persistent storage data in a local MySQLite database that would survive a browser crash. Cross-platform, open architecture, Rich Internet Applications (RIA), with functionality as smooth and usable as a desktop application, were becoming a reality.

RIA were all the rave when LibLime was selected as a 2007 Google Summer of Code mentor, and thus was born ‡biblios, an open-source web-based metadata editor. Last year LibLime released ‡biblios under the GPL and the software is freely available for download from http://biblios.org. If you’re interested in learning more about ‡biblios, checkout the Website. There’s also a great article written by Chris Catalfo, the lead programmer of ‡biblios in the latest Code4lib Journal that gives a history and overview of ‡biblios functionality and future directions.

Putting it all together … ‡biblios.NET

‡biblios the editor provides one part of a technology framework for community-maintained data. The other part, a web-scale, production-ready platform where librarians can search, create, share and collaborate, is where ‡biblios.net comes in. ‡biblios.net is the world’s first community-built and maintained database of freely-licensed library records. It’s more than a cataloging editor, it’s a comprehensive cataloging productivity suite including the following components:

• A rich cataloging editor with support for both original and copy-cataloging;
• An integrated metasearch engine for finding records within the ‡biblios.net database, and any other database that supports Z39.50;
• A Search Target Repository, seeded with over 2,000 Z39.50 servers, allowing users to find, create and share useful Z39.50 targets;
• Social cataloging features like Forums, and Private Messaging;
• Context-sensitive, field-specific help, available within the editor itself;
• Community-maintained, wiki-style write access to the ‡biblios.net master bibliographic and authority files;
• Future-thinking design, that will easily adapt to changing metadata and search standards, with support for RDA, MODS, Dublin Core and SRU in development;
• Free access for all to ‡biblios.net as a Z39.50 target;
• Free access for all to the entire batch of data via OAI-PHM harvesting and direct download;
• ‡biblios.net Web Services (BWS), a set of open APIs that enable programmers to write applications to interact with the ‡biblios.net database.

Perhaps best of all, not only does ‡biblios.net contain freely-licensed library records, the service itself, including the cataloging editor is made available for use at no cost. That’s right, ‡biblios.net is FREE.

The future is open…

These other types of open, including Open Data, will play a fundamental role in the library of the 21st century. ‡biblios.net isn’t the first such effort, and it won’t be the last. We’re well positioned as an industry to take advantage of new offerings like ‡biblios.net and move toward a more open, collaborative environment for library data and libraries.

I hope you’ll check out ‡biblios.net and let us know what you think. Who knows, maybe you’ll even join us in our quest to build the world’s largest repository of freely-licensed library records.

Sincerely,