Migrating a CDS/ISIS based system to Koha
2004-12-18
Baiju
M
baijum81 AT hailmail.net
2.0.0p2
2004-12-18
sh
Fix some links
2.0.0p1
2004-11-15
bm
Small additions by BM.
2.0.0
2004-04-01
bm
Initial XML markup (by Stephen Hedges) of a wiki page posted
at http://www.saas.nsw.edu.au/koha_wiki/index.php?page=CDS%2FISIS+based+system+to+Koha
on 2004-04-01, with links to program listings expanded.
Using this document
Copyright and License
Copyright 2004 Baiju M baijum81 AT
hailmail.net
This document is related to Koha and is licensed to you under the
GNU General Public License version 2 or later (http://www.gnu.org/licenses/gpl.html).
Koha-related documents may be reproduced and distributed in whole
or in part, in any medium physical or electronic, as long as this
copyright notice is retained on all copies.
You may create a derivative work and distribute it provided that
you:
License the derivative work with this same license, or the
Linux Documentation Project License (http://www.tldp.org/COPYRIGHT.html).
Include a copyright notice and at least a pointer to the license
used.
Give due credit to previous authors and major
contributors.
Commercial redistribution is allowed and encouraged; however, the
author would like to be notified of any such distributions.
Disclaimer
No liability for the contents of this document can be accepted.
Use the concepts, examples and information at your own risk. There may
be errors and inaccuracies that could be damaging to your system.
Although this is highly unlikely, proceed with caution; the author(s) do
not take any responsibility.
All copyrights are held by their respective owners, unless
specifically noted otherwise. Use of a term in this document should not
be regarded as affecting the validity of any trademark or service mark.
Naming of particular products or brands should not be seen as
endorsements.
Introduction
I am doing this migration from CDS/ISIS (version : 1.5) system to
Koha for St. Joseph's College, Devagiri,
Calicut. http://devagiricollege.org/
In the current system it is possible to do circulation and other
basic functionalities of the library.
The work is in progress, so give any pointers if you have.
Migrating Book Data
First I exported Books data into XML using CDS/ISIS export tool
(available in the ISIS Utils menu).
Users of previous CDS/ISIS versions (i.e, 1.3 or before),
export documents data from CDS/ISIS and save it as Common
Communication Format (CCF), which is a standard based on ISO 2709.
Import this ISO data into CDS/ISIS version 1.4 or later. This step
is necessary to export the data as XML, the old versions of ISIS do
not have support for exporting document data as XML directly.
FIXME: ISIS 1.3.x support exporting XML?
Then I wrote a python script (shown below) to convert XML to tab
delimited text format.
This step is not necessary, but you can do some corrections
in data quickly.
Use Emacs macro or Vim recording for quick parsing.
## ciddtf.py : Convert ISIS Document Database to Text File
##Before Parsing :
##Change RECORDX\xc3\x91_5\xc3\xaf\x03\x12 to RECORD
##Change & to and
##fix problems of <Tag_97> manualy.
import libxml2
record = []
def Record(i, node):
sub_node = node.children
while sub_node is not None:
if sub_node.type == "element":
record[i][sub_node.name] = sub_node.get_content()
sub_node = sub_node.next
else:
sub_node = sub_node.next
# Memory debug specific
libxml2.debugMemory(1)
isis_data = libxml2.parseFile ('../xmls/docs.xml')
root = isis_data.getRootElement()
node = root.children
i = 0
while node is not None:
if node.type != "element":
node = node.next
continue
if node is None:
break
if node.name == "RECORD":
##print node.get_content()
record.append({})
Record(i, node)
i = i + 1
node = node.next
else:
print "unhandled node in <isis_xml>: " + node.name
isis_data.freeDoc()
# Memory debug specific
libxml2.cleanupParser()
if libxml2.debugMemory(1) == 0:
print "OK"
else:
print "Memory leak %d bytes" % (libxml2.debugMemory(1))
libxml2.dumpMemory()
####################################################################
record_file = open('../outs/docs.txt', 'w')
tag = ['Tag_4', 'Tag_7', 'Tag_11', 'Tag_12', 'Tag_13', 'Tag_18', 'Tag_19', 'Tag_20', 'Tag_21', 'Tag_24', 'Tag_38', 'Tag_10', 'Tag_114', 'Tag_115', 'Tag_110', 'Tag_5', 'Tag_122', 'Tag_97', 'Tag_151', 'Tag_153', 'Tag_123']
for j in record:
if j.has_key('Tag_691'):
record_file.write(j['Tag_691'])
##Check docs without 114 tag
## if j.has_key('Tag_114'):
## pass
## else:
## print j['Tag_691']
else:
record_file.write('_B_L_A_N_K_')
for h in tag:
if j.has_key(h):
record_file.write('\t'+j[h])
else:
record_file.write('\t_B_L_A_N_K_')
record_file.write('\n')
Here is a sample XML:<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE DATABASE_SJCLDB SYSTEM "SJCLDB.dtd">
<DATABASE_SJCLDB>
<RECORD>
<Tag_691>43009</Tag_691>
<Tag_4>Chiriyute pithamahan: V K N Krutikalekkurichulla paddhanangal</Tag_4>
<Tag_11>DC Books</Tag_11>
<Tag_12>Kottayam</Tag_12>
<Tag_13>1997</Tag_13>
<Tag_18>Roy</Tag_18>
<Tag_19>P C</Tag_19>
<Tag_38>Ed</Tag_38>
<Tag_10>134p</Tag_10>
<Tag_114>894.37 Roy</Tag_114>
<Tag_115>N7</Tag_115>
<Tag_110>GL-M3</Tag_110>
<Tag_122>2000-03-28</Tag_122>
<Tag_151>Rs.40.00</Tag_151>
<Tag_123>Tom</Tag_123>
</RECORD>
<RECORD>
<Tag_691>43018</Tag_691>
<Tag_4>Edanadan pattu</Tag_4>
<Tag_11>Current Books</Tag_11>
<Tag_12>Kottayam</Tag_12>
<Tag_13>1997</Tag_13>
<Tag_18>Sajitha</Tag_18>
<Tag_19>K R</Tag_19>
<Tag_38>Ed</Tag_38>
<Tag_10>160p</Tag_10>
<Tag_114>398.87 Saj</Tag_114>
<Tag_115>N7</Tag_115>
<Tag_110>GL-3</Tag_110>
<Tag_122>2000-03-28</Tag_122>
<Tag_151>Rs.50.00</Tag_151>
<Tag_123>Tom</Tag_123>
</RECORD>
<RECORD>
<Tag_691>38980</Tag_691>
<Tag_4>Plays</Tag_4>
<Tag_11>EWP</Tag_11>
<Tag_12>New York</Tag_12>
<Tag_13>1989</Tag_13>
<Tag_18>O'Neill</Tag_18>
<Tag_19>Eugene</Tag_19>
<Tag_114>O7,2 One</Tag_114>
<Tag_115>M9.1</Tag_115>
<Tag_122>1996-06-27</Tag_122>
</RECORD>
<RECORD>
<Tag_691>44579</Tag_691>
<Tag_4>Arangukaanatha Nadan</Tag_4>
<Tag_11>Current</Tag_11>
<Tag_12>Thrissur</Tag_12>
<Tag_13>1998</Tag_13>
<Tag_18>Thikkodian</Tag_18>
<Tag_10>396p</Tag_10>
<Tag_114>894.8 Thi/Ara</Tag_114>
<Tag_115>N8</Tag_115>
<Tag_110>GL-M8</Tag_110>
<Tag_122>2001-06-19</Tag_122>
<Tag_151>Rs.135.00</Tag_151>
<Tag_123>Tom</Tag_123>
</RECORD>
<RECORD>
<Tag_691>16469</Tag_691>
<Tag_4>Sartre: A collection of critical essays</Tag_4>
<Tag_11>Prentice Hall</Tag_11>
<Tag_12>USA</Tag_12>
<Tag_13>1965</Tag_13>
<Tag_18>Kern</Tag_18>
<Tag_19>Edith</Tag_19>
<Tag_38>Ed</Tag_38>
<Tag_10>180p</Tag_10>
<Tag_114>O-,4 M79</Tag_114>
<Tag_115>J5</Tag_115>
<Tag_110>Twentieth century views</Tag_110>
<Tag_122>1998-09-26</Tag_122>
</RECORD>
<RECORD>
<Tag_691>42068</Tag_691>
<Tag_4>Internet for beginners</Tag_4>
<Tag_11>Icon Books</Tag_11>
<Tag_12>Australia</Tag_12>
<Tag_13>1997</Tag_13>
<Tag_18>Brunner</Tag_18>
<Tag_19>laurel</Tag_19>
<Tag_20>Jevtic</Tag_20>
<Tag_21>Zoran</Tag_21>
<Tag_10>175p</Tag_10>
<Tag_115>N7</Tag_115>
<Tag_122>1998/09/26</Tag_122>
<Tag_123>CPYR</Tag_123>
</RECORD>
<RECORD>
<Tag_691>42367</Tag_691>
<Tag_4>Text book of Biochemistry with clinical correlations</Tag_4>
<Tag_7>Ed.4</Tag_7>
<Tag_11>Wiley-Liss/John Wiley and Sons</Tag_11>
<Tag_12>New York</Tag_12>
<Tag_13>1997</Tag_13>
<Tag_18>Devlin</Tag_18>
<Tag_19>Thomas M</Tag_19>
<Tag_38>Ed</Tag_38>
<Tag_10>1186p</Tag_10>
<Tag_114>612.015 Dev/KD</Tag_114>
<Tag_115>N7</Tag_115>
<Tag_110>Zoology</Tag_110>
<Tag_122>1998-10-08</Tag_122>
<Tag_97>$.30.95</Tag_97>
</RECORD>
</DATABASE_SJCLDB>
Now parse the tab separated file using a perl script (shown
below) and convert it as MARC 21 or UNIMARC.
Before doing this last step, please study Stephen Hedges'
migration guide here: http://www.kohadocs.org/migrating_to_koha.html
package cmftf;
## Create MARC From Text File
## Copyright 2004 Baiju M <baijum81@hailmail.net>
## This program is licensed under GNU GPL.
use strict;
use MARC::Record;
my $input_file = "../outs/docs.txt";
my $output_file = "../outs/docs.out";
my $repeated_items = " ";
open(INFILE, $input_file) or die "Can't open $input_file: $!";
open(OUTFILE, ">>", $output_file) or die "Can't open $output_file: $!";
my $c = 0;
while (<INFILE>) {
$c++;
my $biblionumber_ftf_691_090_c, my $title_ftf_4_245_a, my $edition_ftf_7_250_a, my $publisher_ftf_11_260_b, my $publishcountry_ftf_12_260_a, my $publishyear_ftf_13_260_c, my $author1ee_ftf_18_100_a, my $author1se_ftf_19_100_a, my $author2ee_ftf_20_700_a, my $author2se_ftf_21_700_a, my $adiauthors_ftf_24_700_a, my $role_ftf_38_590_a, my $phydescr_ftf_10_300_a, my $classno_ftf_114_852_k, my $classno_ftf_115_852_h, my $keyword_ftf_5_520_a, my $voldetails_ftf_110_440_v, my $dateofentry_ftf_122_952_v, my $notes_ftf_97_500_a, my $pubprice_151_952_r, my $currency_153_590_b, my $reccreated_123_508_a, my $remainder;
($biblionumber_ftf_691_090_c, $title_ftf_4_245_a, $edition_ftf_7_250_a, $publisher_ftf_11_260_b, $publishcountry_ftf_12_260_a, $publishyear_ftf_13_260_c, $author1ee_ftf_18_100_a, $author1se_ftf_19_100_a, $author2ee_ftf_20_700_a, $author2se_ftf_21_700_a, $adiauthors_ftf_24_700_a, $role_ftf_38_590_a, $phydescr_ftf_10_300_a, $classno_ftf_114_852_k, $classno_ftf_115_852_h, $keyword_ftf_5_520_a, $voldetails_ftf_110_440_v, $dateofentry_ftf_122_952_v, $notes_ftf_97_500_a, $pubprice_151_952_r, $currency_153_590_b, $reccreated_123_508_a, $remainder) = split(/\t/, $_, 23);
if (($biblionumber_ftf_691_090_c !~ /[^\d]/) and ($title_ftf_4_245_a !~ /_B_L_A_N_K_/) and ($repeated_items !~ /$biblionumber_ftf_691_090_c/) ) {
$repeated_items .= " $biblionumber_ftf_691_090_c";
my @biblionumber_array;
open(INF, $input_file) or die "Can't open $input_file: $!";
my $i = 1;
while (<INF>) {
$i++;
if ($i >= $c) {
my $biblionumber_ftf_691_090_c2, my $title_ftf_4_245_a2, my $edition_ftf_7_250_a2, my $publisher_ftf_11_260_b2, my $publishcountry_ftf_12_260_a2, my $publishyear_ftf_13_260_c2, my $author1ee_ftf_18_100_a2, my $author1se_ftf_19_100_a2, my $author2ee_ftf_20_700_a2, my $author2se_ftf_21_700_a2, my $adiauthors_ftf_24_700_a2, my $role_ftf_38_590_a2, my $phydescr_ftf_10_300_a2, my $classno_ftf_114_852_k2, my $classno_ftf_115_852_h2, my $keyword_ftf_5_520_a2, my $voldetails_ftf_110_440_v2, my $dateofentry_ftf_122_952_v2, my $notes_ftf_97_500_a2, my $pubprice_151_952_r2, my $currency_153_590_b2, my $reccreated_123_508_a2, my $remainder2;
($biblionumber_ftf_691_090_c2, $title_ftf_4_245_a2, $edition_ftf_7_250_a2, $publisher_ftf_11_260_b2, $publishcountry_ftf_12_260_a2, $publishyear_ftf_13_260_c2, $author1ee_ftf_18_100_a2, $author1se_ftf_19_100_a2, $author2ee_ftf_20_700_a2, $author2se_ftf_21_700_a2, $adiauthors_ftf_24_700_a2, $role_ftf_38_590_a2, $phydescr_ftf_10_300_a2, $classno_ftf_114_852_k2, $classno_ftf_115_852_h2, $keyword_ftf_5_520_a2, $voldetails_ftf_110_440_v2, $dateofentry_ftf_122_952_v2, $notes_ftf_97_500_a2, $pubprice_151_952_r2, $currency_153_590_b2, $reccreated_123_508_a2, $remainder2) = split(/\t/, $_, 23);
if (($biblionumber_ftf_691_090_c2 !~ /[^\d]/) and ($title_ftf_4_245_a2 !~ /_B_L_A_N_K_/) and ($title_ftf_4_245_a2 eq $title_ftf_4_245_a) and ($author1ee_ftf_18_100_a2 eq $author1ee_ftf_18_100_a)){
print "$biblionumber_ftf_691_090_c:$biblionumber_ftf_691_090_c2\t";
$repeated_items .= " $biblionumber_ftf_691_090_c2";
push @biblionumber_array, $biblionumber_ftf_691_090_c2;
}
}
}
print "\n";
close(INF);
my $record = MARC::Record->new();
if ($author1ee_ftf_18_100_a =~ /_B_L_A_N_K_/) {
$author1ee_ftf_18_100_a = "";
}
if ($author1se_ftf_19_100_a =~ /_B_L_A_N_K_/) {
$author1se_ftf_19_100_a = "";
}
my $author1_ftf = join(" ", $author1se_ftf_19_100_a, $author1ee_ftf_18_100_a);
if ($author1_ftf ne "") {
my $author1 = MARC::Field->new(
'100','1','',
a => $author1_ftf
);
$record->add_fields($author1);
}
if ($title_ftf_4_245_a !~ /_B_L_A_N_K_/) {
my $title = MARC::Field->new(
'245','1','2',
a => $title_ftf_4_245_a
);
$record->add_fields($title);
}
if ($edition_ftf_7_250_a !~ /_B_L_A_N_K_/) {
my $edition = MARC::Field->new(
'250','','',
a => $edition_ftf_7_250_a
);
$record->add_fields($edition);
}
if ($publishcountry_ftf_12_260_a !~ /_B_L_A_N_K_/) {
my $publishcountry = MARC::Field->new(
'260','','',
a => $publishcountry_ftf_12_260_a
);
$record->add_fields($publishcountry);
}
if ($publisher_ftf_11_260_b =~ /_B_L_A_N_K_/) {
$publisher_ftf_11_260_b = "";
}
if ($publishyear_ftf_13_260_c =~ /_B_L_A_N_K_/) {
$publishyear_ftf_13_260_c = "";
}
my $publisher = MARC::Field->new(
'260','','',
b => $publisher_ftf_11_260_b,
c => $publishyear_ftf_13_260_c
);
$record->add_fields($publisher);
if ($phydescr_ftf_10_300_a =~ /_B_L_A_N_K_/) {
$phydescr_ftf_10_300_a = "";
}
my $phydescr = MARC::Field->new(
'300','','',
a => $phydescr_ftf_10_300_a,
f => 'BOOK'
);
$record->append_fields($phydescr);
if ($voldetails_ftf_110_440_v !~ /_B_L_A_N_K_/) {
my $voldetails = MARC::Field->new(
'440','','3',
v => $voldetails_ftf_110_440_v
);
$record->add_fields($voldetails);
}
if ($notes_ftf_97_500_a !~ /_B_L_A_N_K_/) {
my $notes = MARC::Field->new(
'500','','',
a => $notes_ftf_97_500_a
);
$record->add_fields($notes);
}
if ($reccreated_123_508_a !~ /_B_L_A_N_K_/) {
my $reccreated = MARC::Field->new(
'508','','',
a => $reccreated_123_508_a
);
$record->add_fields($reccreated);
}
if ($keyword_ftf_5_520_a !~ /_B_L_A_N_K_/) {
my $keyword = MARC::Field->new(
'520','','',
a => $keyword_ftf_5_520_a
);
$record->add_fields($keyword);
}
if ($role_ftf_38_590_a =~ /_B_L_A_N_K_/) {
$role_ftf_38_590_a = '';
}
if ($currency_153_590_b =~ /_B_L_A_N_K_/) {
$currency_153_590_b = '';
}
my $role = MARC::Field->new(
'590','','',
a => $role_ftf_38_590_a,
b => $currency_153_590_b
);
$record->add_fields($role);
if ($author2ee_ftf_20_700_a =~ /_B_L_A_N_K_/) {
$author2ee_ftf_20_700_a = "";
}
if ($author2se_ftf_21_700_a =~ /_B_L_A_N_K_/) {
$author2se_ftf_21_700_a = "";
}
if ($adiauthors_ftf_24_700_a =~ /_B_L_A_N_K_/) {
$adiauthors_ftf_24_700_a = "";
}
my $adiauthors_ftf = join(" ", $author2se_ftf_21_700_a, $author2ee_ftf_20_700_a, $adiauthors_ftf_24_700_a);
if ($adiauthors_ftf ne "") {
my $adiauthors = MARC::Field->new(
'700','1','',
a => $adiauthors_ftf
);
$record->add_fields($adiauthors);
}
if ($classno_ftf_114_852_k =~ /_B_L_A_N_K_/) {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
my $classno = MARC::Field->new(
'852','','',
k => '999.9999'
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /^\d\d\d/) {
my $classno = MARC::Field->new(
'852','','',
k => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
else {
my $classno = MARC::Field->new(
'852','','',
k => '999.9998',
h => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
}
}
else {
if ($classno_ftf_114_852_k =~ /^\d\d\d/) {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
$classno_ftf_115_852_h = "";
}
my $classno = MARC::Field->new(
'852','','',
k => $classno_ftf_114_852_k,
h => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /^\d\d\d/) {
my $classno = MARC::Field->new(
'852','','',
k => $classno_ftf_115_852_h,
h => $classno_ftf_114_852_k
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
$classno_ftf_115_852_h = "";
}
my $classno = MARC::Field->new(
'852','','',
k => '999.9997',
h => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
}
}
if ($pubprice_151_952_r =~ /_B_L_A_N_K_/) {
$pubprice_151_952_r = '';
}
if ($dateofentry_ftf_122_952_v =~ /_B_L_A_N_K_/) {
$dateofentry_ftf_122_952_v = '';
}
my @biblionumber_array_tmp = sort @biblionumber_array;
foreach (@biblionumber_array_tmp) {
my $biblionumber = $_;
my $barcode = MARC::Field->new(
'952','','',
b => "MAIN",
d => "MAIN",
p => $biblionumber,
r => $pubprice_151_952_r,
u => $biblionumber,
v => $dateofentry_ftf_122_952_v
);
$record->add_fields($barcode);
}
#print "\n";
print OUTFILE $record->as_usmarc();
}
else {
#print $_;
}
}
close(OUTFILE);
And finally use koha's bulkmarcimport.pl
script to convert this to SQL.
Migrating Member Data
To convert member details to Koha,
first I exported member database into XML, then wrote a python script
to convert it to tab separated text file:##cimdtf.py : Convert ISIS Member Database to Text File
##Before Parsing :
##Change & to and
##fix problems of <Tag_*> manualy.
##tag = ['Tag_201', 'Tag_202', 'Tag_203', 'Tag_216', 'Tag_217']
##Tag_201 : Name -> firstname
##Tag_202 : Membership number -> cardnumber
##Tag_203 : Division -> streetaddress
##Tag_216 : Status (UG,PG, etc.) -> categorycode
##Tag_217 : Joining date -> dateenrolled
## CREATE TABLE borrowers (
##01 borrowernumber int(11) NOT NULL auto_increment,
##02 cardnumber varchar(16) NOT NULL default '', ##Tag_202
##03 surname text NOT NULL, ##Tag_201
##04 firstname text NOT NULL, ##' '
##05 title text, ##' '
##06 othernames text, ##''
##07 initials text NOT NULL, ##''
##08 streetaddress text NOT NULL, ##Tag_203
##09 suburb text, ##NULL
##10 city text NOT NULL, ##Tag_203 (Use this?)
##11 phone text NOT NULL, ##''
##12 emailaddress text, ##''
##13 faxnumber text, ##''
##14 textmessaging text, ##''
##15 altstreetaddress text, ##NULL
##16 altsuburb text, ##NULL
##17 altcity text, ##NULL
##18 altphone text, ##''
##19 dateofbirth date default NULL, ##'0000-00-00'
##20 branchcode varchar(4) NOT NULL default '', ##'MAIN'
##21 categorycode char(2) default NULL, ##Tag_216
##22 dateenrolled date default NULL, ##Tag_217
##23 gonenoaddress tinyint(1) default NULL, ##NULL
##24 lost tinyint(1) default NULL, ##NULL
##25 debarred tinyint(1) default NULL, ##NULL
##26 studentnumber text, ##NULL
##27 school text, #NULL
##28 contactname text, ##' '
##29 borrowernotes text, ##''
##30 guarantor int(11) default NULL, ##NULL
##31 area char(2) default NULL, ##'' or NULL
##32 ethnicity varchar(50) default NULL, ##'' or NULL
##33 ethnotes varchar(255) default NULL, ##'' or NULL
##34 sex char(1) default NULL, ##NULL
##35 expiry date default NULL, ##'0000-00-00'
##36 altnotes varchar(255) default NULL, ##''
##37 altrelationship varchar(100) default NULL, ##'workplace'
##38 streetcity text, ##''
##39 phoneday varchar(50) default NULL, ##''
##40 preferredcont char(1) default NULL, ##NULL
##41 physstreet varchar(100) default NULL, ##''
##42 password varchar(30) default NULL, ##NULL
##43 flags int(11) default NULL, ##NULL
##44 userid varchar(30) default NULL, ##NULL
##45 homezipcode varchar(25) default NULL, ##''
##46 zipcode varchar(25) default NULL, ##''
##47 UNIQUE KEY cardnumber (cardnumber),
##48 KEY borrowernumber (borrowernumber)
## ) ;
import libxml2
record = []
def Record(i, node):
sub_node = node.children
while sub_node is not None:
if sub_node.type == "element":
record[i][sub_node.name] = sub_node.get_content()
sub_node = sub_node.next
else:
sub_node = sub_node.next
## Memory debug specific
libxml2.debugMemory(1)
isis_data = libxml2.parseFile ('../xmls/members.xml')
root = isis_data.getRootElement()
node = root.children
i = 0
while node is not None:
if node.type != "element":
node = node.next
continue
if node is None:
break
if node.name == "RECORD":
##print node.get_content()
record.append({})
Record(i, node)
i = i + 1
node = node.next
else:
print "unhandled node in <isis_xml>: " + node.name
isis_data.freeDoc()
# Memory debug specific
libxml2.cleanupParser()
if libxml2.debugMemory(1) == 0:
print "OK"
else:
print "Memory leak %d bytes" % (libxml2.debugMemory(1))
libxml2.dumpMemory()
####################################################################
output_file = open('../outs/members.txt', 'w')
tag = ['Tag_202', 'Tag_203', 'Tag_216', 'Tag_217']
for j in record:
if j.has_key('Tag_202') and j.has_key('Tag_201') and j.has_key('Tag_216'):
output_file.write('\t') ##01
output_file.write(j['Tag_202']+'\t') ##02
output_file.write(j['Tag_201']+'\t') ##03
output_file.write(' \t') ##04
output_file.write(' \t') ##05
output_file.write('\t') ##06
output_file.write('\t') ##07
if j.has_key('Tag_203'): ##08
output_file.write(j['Tag_203']+'\t')
else:
output_file.write('\t')
output_file.write('NULL\t') ##09
output_file.write('\t') ##10
output_file.write('\t') ##11
output_file.write('\t') ##12
output_file.write('\t') ##13
output_file.write('\t') ##14
output_file.write('NULL\t') ##15
output_file.write('NULL\t') ##16
output_file.write('NULL\t') ##17
output_file.write('\t') ##18
output_file.write('0000-00-00\t') ##19
output_file.write('MAIN\t') ##20
if j.has_key('Tag_216'): ##21
output_file.write(j['Tag_216']+'\t')
else:
output_file.write('OT\t')
pass
if j.has_key('Tag_217'): ##22
output_file.write(j['Tag_217']+'\t')
else:
output_file.write('0000-00-00\t')
pass
output_file.write('NULL\t') ##23
output_file.write('NULL\t') ##24
output_file.write('NULL\t') ##25
output_file.write('NULL\t') ##26
output_file.write('NULL\t') ##27
output_file.write(' \t') ##28
output_file.write('\t') ##29
output_file.write('NULL\t') ##30
output_file.write('NULL\t') ##31
output_file.write('NULL\t') ##32
output_file.write('NULL\t') ##33
output_file.write('M\t') ##34
output_file.write('0000-00-00\t') ##35
output_file.write('NULL\t') ##36
output_file.write('workplace\t') ##37
output_file.write('NULL\t') ##38
output_file.write('NULL\t') ##39
output_file.write('NULL\t') ##40
output_file.write('NULL\t') ##41
output_file.write('NULL\t') ##42
output_file.write('NULL\t') ##43
output_file.write('NULL\t') ##44
output_file.write('NULL\t') ##45
output_file.write('NULL\t') ##46
output_file.write('\n')
Later to load it into MySQL:
LOAD DATA LOCAL INFILE "members.txt" INTO TABLE
borrowers;