Integrated Library System Renewal

Last updated September 27, 2014.

In addition to the general undesirability of supporting multiple systems, the technical platform for Horizon became more expensive and more difficult to support as time went on. The Sybase license required an AIX platform and while that had previously been used extensively by many other university applications, thereby reducing everyone’s annual maintenance, most of those applications migrated to more cost-effective hardware and software. As a result, the Library’s annual maintenance for Sybase doubled. The only other relational database management system supported by Horizon is SQL Server and this was not considered a good choice for the size of the Library’s database or for the environment where we wanted to run it.

Another problem was that the Horizon database was never upgraded to Unicode and this caused an increasing number of issues as the Library has an extensive collection of titles in non-Roman alphabets. The Library had to run imported record files through conversion programs; while these generally worked, the pre-Unicode standards applicable to Horizon did not always result in completely correct translations. In addition, SirsiDynix, following the demise of Horizon 8.0/Corinthian (which was Unicode based) announced that it would NOT convert Horizon 7.x to Unicode. The only option was to move to the vendor’s Symphony system, which was Unicode-compliant. However, that system was – following an assessment by Library staff – more or less functionally equivalent to Horizon 7.x; the complexity and cost of a system migration could not be justified if the new system was merely “equivalent” to the old, so this was not considered a good solution.

Environmental scan of commercial systems

Last updated September 27, 2014.

Following the halt of Horizon 8.0/Corinthian development, the Library began an immediate scan of the ILS environment in early 2007. To do a quick survey of possible replacements, the Library’s systems staff arranged for conference calls with the University of North Carolina at Chapel Hill to discuss their implementation of Innovative Interfaces’ Millennium and with Boston College Library for a discussion of its implementation of Ex Libris’ Aleph system. A tabulation of the results of those discussions is at: http://www.lib.uchicago.edu/staffweb/depts/ils/projects/ilsreplacement/systemsevaluation.html.

Contacts were also made with Oxford University and New York University about their work with VTLS on its new system, but the discussions led to the conclusion that this was not a production-ready option.

Of all the options available at that time, only Aleph seemed reasonable given our requirements, and Ex Libris did an on-site, day-long demo for the Library. Its preliminary, informal price quote was surprisingly low, but a later formal quote was significantly higher. In the end, the Library could not justify such a significant investment on a system that, while meeting most of the functional requirements, would not give the Library a more technologically up-to-date platform than it already had.

Ex Libris invited the Library to be a development partner on what eventually became Alma, its next generation ILS. However, the Library was reluctant to commit to another vendor’s new system development project. The fact that Ex Libris’ ownership changed in 2007 (bought by a private equity firm, as SirsiDynix had been) was also not encouraging. So the Library declined to participate.

By this time, other developments were suggesting other alternatives.

Requirements documents from Library functional groups

Last updated September 27, 2014.

Around 2007, open source ILS products were just beginning to attract attention as alternatives to the commercial products. Evergreen and Koha were the open source library systems that seemed to have the most promise, and information about both were actively collected by the Library. Since systems staff were aware that any new ILS would undoubtedly have certain functional gaps, groups of Library staff were asked to identify the core functional requirements for their area. Work had already begun on this analysis during the Horizon 8.0/Corinthian development as it was necessary to enumerate critical requirements in order to verify the system would be able to replace existing systems. Before doing a “gap analysis” on the existing open source systems, we asked staff to formalize these lists of requirements. These lists can be found in a chart that links to Word documents at: http://www.lib.uchicago.edu/staffweb/depts/ils/projects/ilsreplacement/.

Technical investigation of open source

Last updated September 27, 2014.

Investigation of Evergreen proceeded with discussions with its developers and some academic libraries including McMaster University and Project Conifer, a group of academic libraries in Ontario that were considering developing Evergreen sufficiently to enable its use in academic libraries. While Evergreen was developed by and for a consortium of Georgia public libraries, it seemed to have the potential of being scalable and workable as a platform for development of additional features. It was promising enough and mature enough that the Library’s systems staff installed the software and loaded a full copy of the Library database to allow staff to do a gap analysis. Results of that analysis can be found in the chart at: http://www.lib.uchicago.edu/staffweb/depts/ils/projects/ilsreplacement/ with reports by various functional groups. Note that there are comments that sometimes give an assessment of what it would take to remedy a gap. [A gap assessment of Koha was not done to this level of detail so that column does not contain this type of report]. Evergreen looked promising but it definitely lacked some features essential for academic libraries. For instance, academics have a changing patron file that needs to be updated to reflect enrollment and library privileges; fixed due dates are common; and the capability to recall items is often required. Interface with various university systems is also needed, e.g., the ability to export voucher data to university payment systems.

The library sent Stuart Miller to the VALE (NJ) “Next Generation Academic Library System Symposium” in March 2008, which discussed Evergreen, Koha and some other library open source systems, including VuFind. It was openly acknowledged that SirsiDynix’s decision to abandon Horizon 8.0/Corinthian had created a sudden increase of interest in open source products among libraries.

Contact was also made with WALDO, a consortium of fifteen academic libraries in the New York area (the largest being St. John’s in Queens, NY) that was implementing Koha. LibLime, the support company and the North American release manager for Koha, had completed several enhancement projects funded by WALDO; St. John’s was then in production. A conference call with Joshua Ferrero and John Rose of LibLime was held on April 10, 2008. (See “Some Basics on Koha Discussion with LibLime April 2008” on http://www.lib.uchicago.edu/staffweb/depts/ils/projects/ilsreplacement/.)

Koha’s drawbacks were found to be primarily technical – there were questions about its scalability to much larger databases. No large ARL library had been involved, and there was a suggestion the software might need to branch to support ARL library workflows. Library systems staff continued to monitor developments, but it was eventually dropped as an option. It was considered better to wait for changes to support academic library work to be completed before expending the effort for a complete evaluation that involved installing a local copy with the full library database. By the time those changes were available, the library had already decided to participate in the Kuali OLE project.

In 2009, the Library also thought it should review the state of the new OCLC Worldcat library management system; a summary report can be found at: http://www.lib.uchicago.edu/staffweb/depts/ils/projects/ilsreplacement/. Due to many non-existent features at that time, a detailed gap analysis was not thought to be necessary. In addition, there were doubts that this new product could achieve acceptable performance levels, and the Library’s past experience with the quality of OCLC support and services led to skepticism about its ability to provide even an acceptable level of support for a mission-critical service such as an ILS. The fact that the library is open until 1 AM weekdays and is heavily used late at night and on weekends tends to influence decisions about whether trying to use a hosted service makes sense. Library systems staff do provide support nights and weekends, and it was felt this level of support would be difficult to afford via a hosted service.

Invitation to participate in OLE project phase 1

Last updated September 27, 2014.

Meanwhile in 2008, the Andrew W. Mellon Foundation funded a project to design an “Open Library Environment” and work began in August 2008. University of Chicago participated in this project, along with several other large academic libraries, and hosted one of the regional meetings in December 2008. Documents from this meeting including a useful OLE Overview presented by Jim Mouw, Associate Director for Collection Services, University of Chicago Library, can be found at: http://www.lib.uchicago.edu/staffweb/depts/ils/projects/ilsreplacement/OLEUChicworkshop.html. Work on the first phase of this project included training in Business Process Management and production of Tasks and Process Maps for each module of a library system. This resulted in a design document that was presented to Mellon in early summer of 2009.

Interest in OLE also came from Greg Jackson, then director of the University’s IT Services. The OLE Project had discussed the need for a new ILS to work at an enterprise level, e.g., a library’s need for patron data would be satisfied by being able to pull data from a campus database rather than having to maintain a set of patron records like all current ILS systems. Inspired by a conversation with Library Director Judi Nadler, Jackson raised this issue at a meeting of the Common Solutions Group (CSG – http://www.stonesoup.org/ – a network of campus CIOs). Clearly, an “enterprise level” ILS capable of direct interface with other campus applications would require cooperation and input beyond libraries. CSG, while interested, did not “adopt” this as something that the group could fully support, not because it was undesirable, but because most institutions were just not ready for it. Any OLE design would need to recognize this. This did prompt OLE to think that repurposing what already existed might also be the most practical way to approach a new ILS, e.g., taking Evergreen and adding development suitable for academics might be a possibility (although after review, OLE decided to use Kuali Rice instead). Also, OLE began to consider whether or not it should ultimately become a Kuali Foundation project, partly in order to facilitate cooperation with other open source campus application developers.

Decision to become Kuali OLE partner

Last updated September 27, 2014.

At that point, a “build” project for OLE had not been funded, but was in the offering. The Library was concerned at that time that the Horizon system might not be supported by SirsiDynix beyond 2013. [Note: As it turned out, Horizon is still supported by SirsiDynix, and there is, as of March 2014, no announced “end of life” for Horizon.]

Because the library was not ready to choose a true replacement system for the ILS, a “bridge” solution was proposed in October 2009. The Bridge to the Future recommendation (http://www.lib.uchicago.edu/staffweb/depts/ils/projects/ilsreplacement/) summarized the possible options: to implement Evergreen; stay on Horizon as is; or upgrade to Sybase 15 and upgrade to Horizon 7.5.1. It was later found that the cost for the Sybase license was half what was originally estimated, and that final option was eventually implemented. An added concern for the Library at that time was the construction of the Mansueto Library planned to open in 2011. The Mansueto Library was built to house lesser used portions of the collection in a structure that is physically connected to the Regenstein Library. Transfer of parts of the collection to this library allowed newer materials to continue to be shelved in the open stacks, while avoiding the problems associated with offsite storage. There was a requirement that the automated storage and retrieval system in Mansueto interface with the Library system to allow users to seamlessly request items in storage. This would have to be done first for Horizon and then with whatever its replacement would be.

In order to apply for Mellon Foundation funding to build the OLE software, it was necessary for a group of libraries to agree to be founding partners and to contribute matching funds. In 2009, the Library decided to become a founding partner, and the grant proposal to build the OLE software was funded by Mellon to run from July 2010 to June 2012 (additional funding was later secured from Mellon through 2014).

The OLE project joined the Kuali Foundation in December 2009. The Kuali Foundation provides the legal and financial framework that is needed to sustain an open source project, and because all of the Kuali projects that come out of the academic environment, it is set up in a way that works well for an academic library.

As with all previous efforts concerning system replacements, the Library’s systems staff recruited assistance from the University’s IT Services department in considering open source software. The Kuali project was already being monitored because of the university systems that fall under its umbrella. While the University of Chicago had no plans to implement other Kuali software at this time, they are considered viable options. Some work was moving forward by the identity management staff in IT Services to discuss a potential open source identity management system that might be developed.

Decision to become an early adopter

Last updated September 27, 2014.

With the decision to become a full partner in the “build” of OLE, the Library had a plan to implement OLE in the summer of 2013. It was thought that the first usable version of OLE would be released in the summer of 2012, and then it would take roughly a year to install, customize, convert data and integrate the system. (Note that a full year was projected because the software would be so new; later implementers should be able to do this in less time.) Not too surprisingly, there were development delays and setbacks, so the Library’s projected implementation date is now July 2014.

The Library was willing to become one of the first OLE adopters for a number of reasons. First, the clock was running out for support for the hardware and software of the legacy systems. In addition to annual support costs for the two systems and the increasing costs for the Sybase license, the Horizon AIX server would soon need to be replaced. To migrate to a more supportable hardware platform would require another license upgrade to run on Linux and that would be costly. As it turned out, the Library had to replace the HIP server in 2013 because of its advanced age, a project the Library hoped to have avoided.

Second, the Library, as an early adopter, would be able to work directly with the developers to ensure that OLE would work with its large database and that its essential requirements (referred to as its “drop dead” list) would be met. In addition, the Library was extremely eager to move to a Unicode-based system; this basic lack in Horizon was contributing more and more to the overhead support costs.

While any system migration is challenging and complex, the Library believes a migration here is somewhat easier due to a variety of factors. For one, Chicago is far more centralized than many comparable university libraries; all six libraries on campus have all used the same system for many years, and all libraries report into one management structure with one director. In addition, there is a single identity management system for users. Library staff are accustomed to participating in review of specifications and beta testing, having done so for Horizon in 1995 and more recently for Horizon 8.0/Corinthian and AquaBrowser. Staff had long been involved in planning for system migration and many were acting as subject matter experts writing functional requirements for OLE.

Another factor simplifying an OLE implementation was that the Library’s legacy systems had never included Electronic Resource Management (ERM). While OLE includes ERM, it is possible to migrate to OLE and implement the ERM features later.

Discovery Layer Online Catalog

Last updated September 27, 2014.

From its inception, Kuali OLE decided NOT to include an online catalog module in its design, due to the wide availability of several open source interfaces such as Blacklight and VuFind, two of the most popular. It was assumed that Kuali OLE would support any necessary protocols to allow for connectivity and the ability for users to access “my account”-type features (e.g., items checked out, renewals, lists of requested items, updating addresses, etc.).

So beginning in 2011, the Library began a technical evaluation of possible user interfaces as well as initiating an assessment of user requirements.

Under the leadership of Tod Olson, Systems Librarian, Library systems staff did an investigation of Solr based on the open source systems Blacklight and VuFind to assess the technical feasibility of implementing one of these systems with a Kuali OLE-type database. In January 2012, this produced the Solr Catalog Technical Report (see http://www.lib.uchicago.edu/staffweb/groups/disctools/index.html). To quote the summary findings:

  • Implementation language (user interface level). Blacklight is a Ruby on Rails application. We have no local experience with this platform, and there is a significant learning curve. VuFind is a PHP application, the Library has in-house experience with PHP and there is a lot of support on campus for PHP.
  • VuFind has more of our critical features out of the box, notably browse indexes (title, author, subject, and call no.), a built-in architecture for integrating live circulation data, and a built-in authentication framework. Blacklight is working to close this gap, but currently Blacklight supplies fewer needed features than VuFind.

While either platform would be suitable for building our public front-end for Kuali OLE, VuFind appears to be the better match for the Library. VuFind is the recommended platform.

Note that the recommendation was not so much to implement VuFind as is, but to use it as a platform to develop a user interface for Kuali OLE and replace both HIP and Lens.

Meanwhile, a group of library staff was appointed to collect information from users about their needs and desires for an ideal user interface. Following the agile development concept of collecting user stories, the Library did just that with a series of activities involving a cross-section of Library users. Findings were published in a report (see http://www.lib.uchicago.edu/staffweb/groups/cus/). A quote from the full report describes the methodology:

…With the User Stories method, investigators seek to solicit statements of user need. These stories are typically written in plain, rather than technical, language. Stories are constructed so as to avoid complex or interdependent requirements. Stories do not specify a particular solution, but rather they seek simply to describe the need.

The project team began by reviewing existing sources of data to identify user stories. Sources reviewed include comments from multiple LibQual and Library surveys, user email comments in Knowledge Tracker and Bugzilla, requirements documents from Lens development and from a requirements list developed by Stanford, and from prior usability studies. Approximately one hundred unique stories were drawn from these sources. These stories were categorized, and they informed the design of interview questions and research instruments that were used in subsequent data collection.

Beyond the mining of existing data sources, the project team used several methods to collect data specifically for this study. Library staff conducted twenty individual and group interview sessions, involving a total of twenty-seven participants. Seven interviews were conducted by bibliographers, who recruited faculty from contacts. The remaining interviews were conducted with students from a variety of disciplines and programs, and with one College graduate working as a clinical researcher. These participants were recruited using ads on the UChicago Marketplace site, and on the Library web site, and the Library offered a $15 Amazon.com gift card as an incentive.

Taking these two reports together led the Library to select VuFind as its new “front end” with Kuali OLE as the back end to replace both HIP and Lens. It was decided to introduce VuFind as a “beta test” to library users in early 2014 using the Horizon database. User feedback would be used to fine tune the product. In the meantime, VuFind would be tested against Kuali OLE to ensure that once the latter went into production, VuFind would work with it. Since they would already be familiar with VuFind, library users would not even notice the switch on the “back end” and Library systems staff would have some breathing room between implementing Kuali OLE and implementing VuFind.