Data Migration and Integration Issues


Last updated September 27, 2014.

For bibliographic and holdings data migration, the fact that the library was coming from a non-Unicode system required additional work. Kuali OLE also introduced a new type of holding called an EHolding record. Over time the library had dealt with electronic books, journals and other resources in a variety of ways. In some cases there were 865 fields in the MARC records, and in others there were also copy or item records. When we went to represent these materials in VuFind, it was decided to take advantage of the migration to OLE to regularize how we represented these materials by always using an EHolding record in OLE.

There were some special problems in data conversion due to the fact that we were moving from two separate systems, Horizon and III. These required custom data conversion scripts developed by in-house programmers.

MARC Authorities records were not planned to be supported in Kuali OLE until after the implementation of the early adopter sites. For a number of years, the library had sent new bibliographic records to Backstage Library Works and received corrected bibliographic and new and corrected authority headings. It was necessary to adapt this practice to continue to receive corrections for bibliographic records for loading into Kuali OLE, but to load authority records only into VuFind where they are used for cross references in the browse indexes.

Certain other features were not planned to be part of the initial Kuali OLE release and required that other local custom helper applications be developed. So, for instance, the Library used the spine label printing in Horizon, although not all libraries use that feature of an ILS. Also tracking payments accepted at circulation desks had some limited support in Horizon that was not in the new system, so some supplemental helper applications were required to support local workflows. The Library had developed many customized reports by pointing MS Access databases at the Sybase Relational Database Management System (RDBMS) of the Horizon system. Gradual adaptation of these reports to the Kuali OLE RDBMS is expected to take some work by the staff involved.

In addition, there are critical integrations required for the system to be functional. Specific Kuali OLE application programming interfaces (APIs) were developed to allow integration with the Dematic Automated Storage Retrieval System used in the new Mansueto Library. It was necessary to contract with the vendor for them to rewrite their side of that integration. The Library had moved Course Reserve functions to the Atlas ARES system and written a custom integration for the “place on reserve” and “remove from reserve” functions. Kuali OLE docstore APIs will be used to replace the custom integration at the RDBMS level used for Horizon. We are working with Atlas to implement this connection. NISO Circulation Interchange Protocol (NCIP) messaging to support participation in the UBorrow and Borrow Direct projects were also necessary parts of the project and require testing with the vendor systems. Finally, the ability to continue to extract payment information and format it correctly to load to the University Comptroller’s system in order to pay our vendors was another critical integration that is required for going into production.

It was particularly critical that the VuFind Horizon connector be replaced by an OLE Connector in order to have the public catalog work correctly and provide the My Account features to allow self-service requesting and renewals. Because Villanova, where the primary VuFind development is done, was a Kuali OLE partner, they provided the OLE Connector based on the OLE APIs developed for this purpose.

A Basecamp project was used to track work on these various projects and a snapshot of that project is provided in Appendix C.