MARC Holdings, Koha, and Migration

Thomas Dukleth

This document is related to Koha and is licensed to you under the GNU General Public License version 2 or later (http://www.gnu.org/licenses/gpl.html).

Koha-related documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies.

You may create a derivative work and distribute it provided that you:

  1. License the derivative work with this same license, or the Linux Documentation Project License (http://www.tldp.org/COPYRIGHT.html). Include a copyright notice and at least a pointer to the license used.

  2. Give due credit to previous authors and major contributors.

Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

No liability for the contents of this document can be accepted. Use the concepts, examples and information at your own risk. There may be errors and inaccuracies, that could be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark. Naming of particular products or brands should not be seen as endorsements.

2005-08-21

Revision History
Revision 2.2.32005-08-21

Initial XML markup.


1. Current Koha Holdings Model
2. Recommended Changes to Koha
2.1. Changes to the Koha SQL Data Types
2.2. Koha Template Call Number Use
3. MARC 21 to Koha Holdings Mappings
4. MARC 21 Holdings
4.1. MARC 21 Serials Holdings
5. UNIMARC Holdings
5.1. UNIMARC Serials Holdings
6. Development of Koha Holdings Model

Abstract

Things that you were afraid to ask about MARC holdings in Koha with a MARC 21 to Koha holdings table mapping.

1. Current Koha Holdings Model

Koha has adopted a holdings implementation modelled on the French Recommendation 995, http://www.adbdp.asso.fr/outils/infogestion/r995.htm.  Using this model, holdings information for one record is retained in one and only one repeatable local use field within the bibliographic record.  Using Koha with MARC 21, any single field may be adapted for that purpose.  952 is the Koha default implemented at the largest Koha implementation at the time of this writing, Nelsonville Public Library.

2. Recommended Changes to Koha

2.1. Changes to the Koha SQL Data Types

At the time of this writing, the default SQL sets items.itemcallnumber to VARCHAR(30).  Changing the data type to VARCHAR(100) may be more helpful if you have long call numbers.  Call numbers with lengthy prefix or suffix strings may otherwise be truncated.

Similar changes to support large VARCHAR data types should be made to biblioitems.classification, biblioitems.dewey, and biblioitems.subclass, if you decide to use them.  The current default SQL sets biblioitems.dewey to a poor choice for data type.

2.2. Koha Template Call Number Use

All the templates may be adjusted to use items.itemcallnumber as a sensible choice for MARC format Koha.  If that is done, it is not necessary to map any MARC field/subfield to biblioitems.classification, biblioitems.dewey, or biblioitems.subclass in System Administration : MARC Links.

3. MARC 21 to Koha Holdings Mappings

Note

Corrected generally, and with reference to information about Koha tables that Chris Cormack and Paul Poulain gave to me over IRC.  Thank you Chris and Paul.

Yes, this is missing a mapping for other tables that might be part of the migration process.  Mapping to the Koha items table for MARC 21 holdings is the only issue treated here.

Mapping existing standard data to a MARC 21 local use field is necessary when migrating holdings to Koha.  Using the MARC::Record Perl module or some other tool to remap records prior to importing with bulkmarkimport.pl will most likely be necessary.  If you need assistance beyond what may be provided on the Koha listserve, http://www.koha.org/mailing/, consider options for commercial support, http://www.koha.org/installation/support.html.

The following is based on standard usage of MARC 21 holdings stored in a MARC bibliographic format record.  If you require complex serials holdings records as described in later sections of this document, Koha is not yet ready to manage your holdings.  If that is the case, you may choose to develop the necessary changes locally and contribute them back to the Koha code base or you may choose to contribute support to sponsoring the development of the necessary changes.

If your existing system now stores the required values in another field/subfield location presently, this mapping must be adjusted for your records.  If using the default local use field 952, the following may be a common mapping.  Be certain to construct unique 952 fields populated from the corresponding $t, if present.  Mappings with multiple subfields listed should have all listed subfields mapped to the corresponding local use 952 subfield where the last populated subfield value would overwrite any previous populated value for that local use 952 subfield.

  • Map 852 $t, 876 $t, 877 $t, and 878 $t  to  952 $t for a nonexistent items.copynumber column.  (If you have been using $3, you should find some way of combining $t with $3 to form a unique whole number.) Do not link items.itemnumber to a MARC subfield, not even 952 $t in the MARC to Koha links to avoid possible loss of data.  Koha will assign a unique itemnumber to your items with no help from your old records.  If you need to maintain your existing copy numbers in a Koha table, create a new column in Koha for representing that information, such as items.copynumber, and programme a solution.  You could append copy numbers to an appropriate subfield accompanied by a text label 'copy ' within an existing value such as call number, given below, and suppress appending in cases where only one copy exists.

  • Map 852 $p, 876 $p, 877 $p, and 878 $p  to  952 $p for items.barcode.

  • Map 541 $d; or 876 $d, 877 $d, and 878 $d  to  952 $d for items.dateaccessioned.  Be certain to convert to the date format that you have chosen in Koha System Administration : Preferences : Date Format.

  • Map 850 $a, 852 $a, or 852 $b  to  952 $b for items.homebranch.  Be certain to convert to the 4 character all caps code as defined for Koha branches in Koha System Administration : Branches.

  • Map 541 $h; or 876 $c, 877 $c, and 878 $c  to  952 $o for items.price.

  • Map 365 $b; or 876 $c, 877 $c, and 878 $c  to  952 $u for items.replacementprice.

  • Map 365 $f  to  952 $r for items.replacementpricedate.  Be certain to convert to the date format that you have chosen in Koha System Administration : Preferences : Date Format.

  • Map 852 $b  to  952 $d for items.holdingbranch.  Be certain to convert to the 4 character all capitals code as defined for Koha branches in Koha System Administration : Branches.

  • Map 506 $a, 876 $h, 877 $h, and 878 $h  to  952 $y for items.notforloan.  Be certain to convert to an integer where NULL or 0 is for loan and 1 is not for loan.

  • Map 876 $j, 877 $j, and 878 $j  to  952 $1 for items.itemlost.  Be certain to convert to an integer where NULL or 0 is not lost, 1 is lost and paid or by the party responsible for losing the item, and 2 is lost with no payment yet.

  • Map 876 $j, 877 $j, and 878 $j  to  952 $w for items.withdrawn.  Be certain to convert to an integer where NULL or 0 is not withdrawn and 1 is restricted.

  • Map 852 $k, 852 $h, 852 $i, and 852 $m concatenated together sequentially, with separating spaces to join them where needed, to 952 $h for items.itemcallnumber.

  • Map 506 $a, 876 $j, 877 $j, and 878 $j  to  952 $4 for items.restricted.  Be certain to convert to an integer value where NULL or 0 is not restricted and 1 is restricted.

  • Map 852 $c, 856 $u, 876 $l, 877 $l, and 878 $l  to  952 $c for items.location.

After your mappings are complete, be certain to set them for the Koha items table in System Administration : MARC Links.  There should be no mapping to any items table column not given in the mapping above to avoid possible data corruption with automatically populated key columns or senseless use of orphan columns.

4. MARC 21 Holdings

MARC 21 bibliographic has always provided multiple holdings fields.  MARC 21 holdings format for storing holdings information in a separate record follows the original introduction of the USMARC holdings format in 1996.  Despite the holding standard for MARC 21, some systems have continued to implement nonstandard local use holdings fields and subfields.  Certainly, some information often not considered to apply well to standard holdings fields or not provided in the standard at all requires local use field or subfield implementation.  Koha's use of a local use field for holdings is certainly not exceptional.  At the beginning of 2005, OCLC announced their plan to implement holdings in separate holdings format records by year's end.

In the the MARC 21 bibliographic format, as in the MARC 21 holdings format, copy number $t is the linking subfield for identifying the particular copy across multiple holdings fields.  For those cases where a subset of the copy is required, materials specified $3 is the linking subfield for identifying the subset.  Control number for related bibliographic record 004 in MARC 21 holdings identifies the control number 001 for the corresponding MARC 21 bibliographic record on the same system.  System control number 035 in MARC 21 holdings matches 035 in MARC 21 bibliographic on the same system for identifying the control number 001 for a bibliographic record obtained from another system.

4.1. MARC 21 Serials Holdings

MARC 21 holdings fields from both MARC 21 bibliographic records and MARC 21 holdings records allow storing complex serials holdings information in a format that is easy for machines to parse, although difficult for humans to read.  Display presentation and complementary textual holdings fields allow for easy human readability of complex serials holdings.

Complex serials holdings can be stored in the following fields:

  • Captions and patterns fields 853-855

  • Enumeration and chronology fields 863-865

  • The simpler textual holdings fields 866-868

853-855, 863-865, 866-868, and item information fields 876-878 are linked together by field link and sequence number $8.  If the enumeration and chronology fields 863-865 are not used, then location 852 or textual holdings fields 866-868 are linked to item information fields 876-878 by $3.

The all-holdings-in-one-local-use-field model that Koha uses based on Recommendation 995 only allows for simpler format textual holdings that are easy for humans to read but difficult for machines to parse.  At some future time, when Koha needs to support more complex needs, of academic libraries or other libraries with complex serials holdings, or to provide for the configuration of OpenURL resolvers, this issue should be addressed.

5. UNIMARC Holdings

UNIMARC had omitted holdings support from the bibliographic format.  Implementers used local use fields or some other means for holdings information.  The French government supported Recommendation 995 for public libraries.  In 2004, the IFLA approved a UNIMARC holdings format as a separate record from the linked bibliographic record, http://www.ifla.org/VI/8/projects/UNIMARC-HoldingsFormat04.pdf.  Paul Poulain reports that the UNIMARC holdings format has not yet been adopted by any ILS that he knows.

Related bibliographic control number 004 in UNIMARC holdings identifies the control number 001 for the corresponding UNIMARC bibliographic record on the same system.  Other system control number 035 in UNIMARC holdings matches 035 in UNIMARC bibliographic on the same system for identifying the control number 001 for a bibliographic record obtained from another system.  In accordance with the approval of the UNIMARC holdings format, two fields are being added to the UNMARC bibliographic format.  Location and call number 852 is being added to UNIMARC bibliographic to match the corresponding 252 field in UNIMARC holdings.  Electronic location and access 856 is being added to UNIMARC bibliographic to match the corresponding 252 field in UNIMARC holdings.  Within UNIMARC holdings, copy number $t is the linking subfield for identifying the particular copy across multiple fields.

5.1. UNIMARC Serials Holdings

The UNIMARC holdings format allows storing complex serials holdings information in a format that is easy for machines to parse, although difficult for humans to read.  Display presentation and complementary textual holdings fields allow for easy human readability of complex serials holdings.

Complex serials holdings can be stored in the following fields:

  • Captions and patterns fields 500-502 are optional

  • Enumeration and chronology

    • Basic bibliographic unit 510 is mandatory if applicable

    • The other enumeration and chronology fields 511 and 512 are optional

  • The simpler textual holdings fields 520-521 are optional

500-502, 510-512, and 520-521 are linked together by interfield linking data $6.

Recommendation 995 only allows for simpler format textual holdings that are easy for humans to read but difficult for machines to parse.  At some future time, when Koha needs to support more complex needs of academic libraries or other libraries with complex serials holdings, or to provide for the configuration of OpenURL resolvers, this issue should be addressed.

6. Development of Koha Holdings Model

An account of the development of the Koha holdings model can be found in the "Koha diary" along with much other useful history.  (Hedges, Stephen. A Koha diary : implementing Koha at the Nelsonville Public Library. kohadocs.org. 2005. http://www.kohadocs.org/koha_diary.html.)

The diary describes part of how the default Koha table linkings to MARC fields had originated.  (Poulain, Paul. Re--dumpmarc : date--Wednesday May 21, 2003, 10:02 AM ; to--shedges / from--paul.p.)

Stephen is quoted.  "We've been discussing our item information.  There is _so_ much valuable information in our old system about each item that we don't want to lose, but which doesn't fit into a MARC tag.  Things such as date last seen (which is also in Koha items table) or date last borrowed (also in Koha).  We're very tempted to just write a script to read the item information from our old system and load it directly into the Koha items table."

Paul replied.  "Why don't you add specific non-standard marc tags in the parameters table and map them to the corresponding koha fields ? Note that I agree to say that there are only a few 852 subfields code that are free in MARC21 (1,4,5,7,9,d,o,u,v,w,y if my table is right), but it may be enough."

Local use field 952 had been used for holdings by the ILS that NPL had been using prior to migrating to Koha and was retained for the Koha to MARC items linkings and presented as a default for Koha.  The substitution of a local use field avoids the risk of some possible future assignment of unassigned subfields for those items table columns that are not part of the MARC standard.