Publisher’s version / Version de l'éditeur:
Student Report, 2010-08
READ THESE TERMS AND CONDITIONS CAREFULLY BEFORE USING THIS WEBSITE. https://nrc-publications.canada.ca/eng/copyright
Vous avez des questions? Nous pouvons vous aider. Pour communiquer directement avec un auteur, consultez la première page de la revue dans laquelle son article a été publié afin de trouver ses coordonnées. Si vous n’arrivez pas à les repérer, communiquez avec nous à PublicationsArchive-ArchivesPublications@nrc-cnrc.gc.ca.
Questions? Contact the NRC Publications Archive team at
PublicationsArchive-ArchivesPublications@nrc-cnrc.gc.ca. If you wish to email the authors directly, please see the first page of the publication for their contact information.
NRC Publications Archive
Archives des publications du CNRC
For the publisher’s version, please access the DOI link below./ Pour consulter la version de l’éditeur, utilisez le lien DOI ci-dessous.
https://doi.org/10.4224/17210703
Access and use of this website and the material on it are subject to the Terms and Conditions set forth at
Ice Tank Database
Doyle, C.
https://publications-cnrc.canada.ca/fra/droits
L’accès à ce site Web et l’utilisation de son contenu sont assujettis aux conditions présentées dans le site LISEZ CES CONDITIONS ATTENTIVEMENT AVANT D’UTILISER CE SITE WEB.
NRC Publications Record / Notice d'Archives des publications de CNRC:
https://nrc-publications.canada.ca/eng/view/object/?id=b35570e7-6550-40c2-b538-42bd7ddbc438 https://publications-cnrc.canada.ca/fra/voir/objet/?id=b35570e7-6550-40c2-b538-42bd7ddbc438DOCUMENTATION PAGE
REPORT NUMBER SR-2010-16
NRC REPORT NUMBER DATE
August 2010 REPORT SECURITY CLASSIFICATION
Unclassified
DISTRIBUTION Unlimited TITLE
ICE TANK DATABASE AUTHOR (S)
Christofer Doyle
CORPORATE AUTHOR (S)/PERFORMING AGENCY (S)
Institute for Ocean Technology, National Research Council, St. John’s, NL PUBLICATION
SPONSORING AGENCY(S)
IOT PROJECT NUMBER NRC FILE NUMBER
KEY WORDS
legacy data, searchable database
PAGES iv, 11, App. A-B
FIGS. TABLES
SUMMARY
Approximately four months ago, there was a requirement to transfer legacy data to a modern searchable database. The data is comprised of important information acquired during the testing of boat models in the Ice Tank at the National Research Council’s (NRC), Institute for Ocean Technology (IOT). The data transfer would proceed in two phases. The first phase involved writing a program, using the Python language of computer
programming, that would read the test files collected from the Ice Tank and retain the data in an internal data structure. Phase two would take the program a step further and would store that data within IOT’s Microsoft SQL Server7 searchable database. Upon reviewing the specifications and constraints of the project, phase one is now approximately 95% completed. Phase two is still in progress. Once phase two is completed and the database is transferred, it will allow researchers within IOT to search for any information they require that relates to the tests carried out in the Ice tank.
ADDRESS National Research Council Institute for Ocean Technology Arctic Avenue, P. O. Box 12093 St. John's, NL A1B 3T5
National Research Council Conseil national de recherches Canada Canada Institute for Ocean Institut des technologies Technology océaniques
ICE TANK DATABASE
SR-2010-16
Christofer Doyle
SUMMARY
Approximately four months ago, there was a requirement to transfer legacy data to a modern
searchable database. The data is comprised of important information acquired during the testing
of boat models in the Ice Tank at the National Research Council’s (NRC), Institute for Ocean
Technology (IOT). The data transfer would proceed in two phases. Phase one involved writing a
program, using the Python programming language, that would read the test files collected from
the Ice Tank and retain the data in an internal data structure. Phase two would involve adding a
module to the program that would store the data within IOT’s Microsoft SQL Server7 database.
Upon reviewing the specifications and constraints of the project, phase one is now approximately
95% completed. Phase two is still in progress. Once phase two is completed and the database is
transferred, it will allow researchers within IOT to search for any information they require that
relates to the tests carried out in the Ice tank.
TABLE OF CONTENTS
1.0
Problem Definition... 1
1.1
Problem Statement ... 1
1.2
Constraints & Specifications... 1
2.0
Test Files... 2
2.1
How the Program Read the Files ... 2
2.2
The Complexity of Certain Types of Files ... 4
3.0
The Program... 5
3.1
Programming Difficulties ... 5
3.2
How the Program Was Organized ... 6
4.0
The Searchable Database ... 8
4.1
Purpose... 8
5.0
Conclusion ... 9
5.1
Review ... 9
5.2
Suggestions ... 10
Appendix A
Appendix B
1.0
PROBLEM DEFINITION
1.1
Problem Statement
Ice Tank researchers at IOT required a searchable database that would allow them to
store large amounts of data as well as allow quick and easy access to essential
information acquired during previous testing of boat models.
1.2
Constraints & Specifications
There were several items that needed to be kept in mind when writing the program that
would read the Ice Tank test files. These items would benefit the efficiency and reliability
of the project.
The constraints and specifications of the Program were:
o
Must be simple and easy to read
o
Must have a certain degree of flexibility
o
Must be written in the Python programming language
o
Must be able to read all the different types of test files
o
Must obtain all the necessary data from each file
Each constraint was necessary for its own reason. The program needed to be easy to read
and simple so that future programmers and IOT staff members who open up the program
will be able to understand it. By making it easily legible, they can quickly see the
function of each line of code without needing excessive commenting.
By making the code easy to read and understand, a certain degree of flexibility was
obtained. Now whoever opens the program up in the future, can easily add and remove
lines of code. If, for example, a new type of test file was introduced, future programmers
can open up the program and change or add new lines of code that would allow it to read
these new types of files without too much effort.
The Python programming language has already been used in past projects at IOT and it
only made sense that this project be written in Python. This would make it easier for IOT
employees to have a look at the program and understand it without having to learn a new
programming language.
There were several different types of legacy project files that the program was needed to
be able to read. Files such as Hull Ice Friction (.HIF), Compressive Strength (.CMP),
Flexural Strength (.FLX), etc., are some examples of the test files obtained from the Ice
Tank. In total there were nine different types of files that the program would read in order
to capture all the necessary data.
In some files, there was information that was already in the database. The information
that was not in the database was read by the program and then formatted into an internal
data structure. The information that the new database is already aware of includes titles,
and column headings. This information was already transferred during the design of the
new database.
2.0
TEST FILES
2.1
How the Program Read the Files
Each of the nine test files types included in the database had its own format. Some files
types had data formatted to in a table with column headers. While others simply had the
data in a list format. Since no two types of files had the same format, an individual piece
of code had to be written for each file type. For some files, the program searched for
certain characters, words, or lines in the file and would then start collecting data from that
point on. It would stop collecting data once another line was found. The following is an
example taken from an Elastic Modulus (.MOD) file to demonstrate how the files were
read.
NRC - INSTITUTE FOR OCEAN TECHNOLOGY ARCTIC VESSEL RESEARCH SECTION
ELASTIC MODULUS Test Name: TC_AZI1
Project Number: PJ2409 Date: 02/25/2010 Time: 0835 Location: 40S
Ice thickness: .0205 meter Disc radius: .0600 meter
Load(N) #div Gain Voltage Deflection(m) Lc Elastic Modulus 1.961 5.0 1.000 5.000 .0006162 0.1955 17.83 2.942 6.9 1.000 6.900 .0008504 0.2043 21.25 3.922 9.8 1.000 9.800 .0012078 0.1976 18.60 4.903 12.5 1.000 12.500 .0015405 0.1955 17.83 mean: 0.1982 18.88 s.d.: 0.0042 1.62 n= 4
The above .MOD file was one of the simpler types of files. For this type of file, the
program first went through each line and split the line in two segments based on one of
two symbols. The line would get split into a key and a value based on a “ = ” or “ : ”
symbol. The key would normally be text taken from the left side of the symbol and the
value would be data taken from the right side. For example, a key in the above .MOD file
would be ‘Time’, and its corresponding value would be ‘0835’. Now all the data on lines
that included an “ = ” or a “ : ” symbol, could be captured by creating an empty
dictionary and appending the keys to the dictionary. A dictionary can thought of as a
container used to store a unique set of keys, from which the corresponding values can be
accessed. The program could then call on the keys inside the dictionary to access the
corresponding values.
For the lines that did not include these two symbols, the program had to search for a
different set of characters. For the above .MOD file, the program would search for a line
containing ‘ #div ’. Once that particular line was found, it would start collecting data
immediately from the next line of the file. The values collected were appended to a
dictionary called ‘DATA’. Values would stop being collected once a line containing
“mean:” was found.
As mentioned above, this was one of the simpler types of files. However, there were
some types of files that were much more complicated than this and need more elaborate
coding.
2.2
The Complexity of Certain Types of Files
There were several types of files that needed more attention. The code for these often
took twice as long to write than it did for any other type of file. There were some
characters, or lack of characters in some cases, that made these types of files harder to
deal with. For example, some files would contain two or more “ : ” symbols. For those
files, the line could not simply be split into a single key and a single value. An entirely
new approach had to be taken. The line was split based on a certain point in the line.
Perhaps after a certain word in the line, but most often it was at a certain column in the
line. Using the column approach, the program would split the line into two segments that
each contained a key and a value. The two segments would then be broken down again
and the two keys and their corresponding values were obtained.
The best examples of these types of files were the Ice Sheet Summary (.ISS) files. For the
.ISS files, a separate program was written. It worked much in the same way as the
previous program used for parsing the other files, but it contained changes in the code
that would allow the program to deal with the different format of these files.
The following is a small example of an .ISS file used to demonstrate the differences
between this type of file and the other types.
SEEDING:
---
Air temp.(max/min) C: -20.2/-14.3 Tank water temp. C: 0.14
Seeding completed at 2344 24-FEB-2010 Seed duration: (min) 39.
Seed volume: l 32.8 Seed water temp.: C 29.0
Humidity: tank(%) 74
room(%) 55
GROWTH:
---
Target temp.: C -20.0 Time to target temp. hrs: 0.7
Avg temp. at plateau: C -20.4 Duration of plateau hrs: 3.9
Avg temp. of freeze cycle C -20.0 Duration of freeze cycle hrs: 4.9
Total negative deg. hours 98.1 Thickness at end of freeze:(mm) 14.3
Avg growth rate: (mm/hr) 2.917 Avg growth rate: (mm/fdh) .146
3.0
THE PROGRAM
3.1
Programming Difficulties
Several programming errors were encountered throughout the duration of the project.
Most times the problems were quickly solved, as Python would point out where in the
program the error occurred. If it was simply a syntax error, the problem was solved
immediately. Something as simple as writing ‘valu’ instead of ‘value’ would generate a
syntax error.
Problems that were not classified as syntax error, most likely fell under the category of
logical errors. These often took a longer period of time to solve as Python rarely pointed
out logical errors. Sometimes the program would run but would not parse the required
data. No syntax errors could be found and the part of the code that failed would have to
be reviewed several times before finally finding the logical error. Print statements were
sometimes used to help locate the logical errors.
A classic example of a logical error would be to indent an ‘if’ statement so that it
encompassed the entire program when perhaps you did not want the program to run
through all the ‘if’ statements. For example, if you wrote two ‘if’ statements but wanted
the second statement to apply only to one particular line, then you would have to use
either an ‘elif’ or an ‘else’ statement. The ‘elif’ statement is carried out when the ‘if’
statements above it is false. The ‘else’ statement is like the last resort. If all ‘if’ and ‘elif’
statements above are false, then the ‘else’ is carried out.
Therefore, using a combination of ‘if’, ‘elif’, and ‘else’ statements is the logical approach
to writing a good working program.
3.2
How the Program Was Organized
The program followed a logical sequence. It goes through one set of file types before
moving onto the next type. It starts by parsing .CMP files and then moves on to Density
(.DEN) files and so on. If it encounters a problem in the first set of files, it stops and does
not continue on to the other types of files.
Each file contains a project number and test name. These two pieces of data were used to
organize the collected data into sections. There could be several .CMP files but each of
them with different project numbers and test names. All files that contained the same
project number were grouped together to make the data structure easier to understand and
read. In most cases, one of each type of file contained the same project number so that
one project was made up of 9 different types of files.
The program would initially look for a project number and a test name. Once that data
was found, it would create a dictionary based on the project number and then a sublevel
of the dictionary based on the test name. The data collected from a file would then get
pushed into the correct dictionary. For example, if three files had the project number
‘PJ023’, they would get pushed into the same project dictionary. A new sublevel for that
project dictionary would be created for each of the three files based on their test names.
Once the project dictionary and the test name sublevels were created, all data collected
from the file would be pushed into the correct location. The program ran through all the
types of files unless an error was encountered. If no error were found then the dictionary
would contain all the collected data. The following is a small example of the data in a
dictionary for a project, where ‘PJ2409’ is the project number, ‘TC_AZI1’ is the test
name and the rest is the collected data.
PJ2409 TC_AZI1 Comp 02/25/2010 1045 39N Chatillon (N/V) 196.300 ['0.10', '1.00000', '0.0214', '0.0228', '40.6'] ['0.16', '1.00000', '0.0215', '0.0214', '67.4'] ['0.16', '1.00000', '0.0219', '0.0230', '60.4'] ['0.11', '1.00000', '0.0214', '0.0200', '50.5'] ['0.09', '1.00000', '0.0211', '0.0195', '43.4'] 44.83 5.06 3 Comp 02/25/2010 1047 39S Chatillon (N/V) 196.300 ['0.10', '1.00000', '0.0218', '0.0205', '43.5'] ['0.13', '1.00000', '0.0215', '0.0209', '56.4'] ['0.14', '1.00000', '0.0220', '0.0211', '61.3'] ['0.17', '1.00000', '0.0219', '0.0230', '65.5'] ['0.15', '1.00000', '0.0216', '0.0217', '63.7'] 58.06 8.83 5
4.0
THE SEARCHABLE DATABASE
4.1
Purpose
The searchable database was the ultimate goal of the project. The project was initiated
because the researchers and staff that work in the Ice Tank at IOT required a database
that would be capable of storing large amounts of data obtained during the testing of boat
models. They want the database to be able to store past data as well as collect new data.
They could then search for a certain project number and test name and find the necessary
information that they wished to acquire. The database will also be available to other staff
members at IOT, so it will be given a comprehensive and easy to understand design.
Therefore, any one wishing to gain certain information could log on to the network
containing the database and search for the information they require.
5.0
CONCLUSION
5.1
Review
Overall, phase one of the project has gone well. It is mostly completed and phase two of
the project is also well under way.
The end result of phase one has met the constraints and specifications listed by the client.
A simple program that is easy to read and understand was developed. It uses the Python
programming language and is able to parse the majority of the data from all the different
types of files. The tempering curve data from the .FSC files is the only piece of
information that the program still needs to capture.
The program uses a unique way of reading and gathering the essential information from
each type of file. Sometimes it will split lines based on special characters or words, and
other times it will collect and store data in dictionaries based on a certain line in the file.
For the more complicated files, several line splits are executed before data is stored into a
dictionary.
Using Python as the programming language did prove to be a challenge at times. Most
problems were very easy to fix because they were simple syntax errors. They were often
a misspelled word or an undefined variable. The logical errors took more time to fix and
were more complicated. Once all errors had been corrected, the program ran as it was
meant to run.
5.2
Suggestions
There are only a few ideas that come to mind that could have improved the efficiency of
the project. Although the project went well, these suggestions may have helped when
dealing with more complicated file types.
The .ISS files proved to be quite challenging and took several times longer to parse than
the other files. At first, the .ISS files were parsed similarly to the other files. After several
errors, it became clear that they would have to be treated differently and a new program
was created that would only parse the .ISS files. This approach may have been
convenient when dealing with a couple of the other more complicated file types. Instead
of trying to treat them the same as all the other files, a new program or code could have
been written. This would have made it easier than trying to use a generic piece of code
that treated all the files similarly.
Secondly, all the accumulated data from each type of file should have been reviewed
before moving onto to another type of file. Some data from a couple of files was not
being collected properly even though it seemed like all the data was captured. This error
was only realized after another file had already begun to be parsed. It would have been
more convenient and logical to check that all the data from each file had been captured
properly before moving on. After beginning another file and realizing that the previous
one had not been completed properly, the code had to be changed and it would affect the
file that was currently being worked on. This cost the project a bit of time and could have
been avoided without much effort.
Aside from these two suggestions, the rest of the project went smooth. The
majority of errors that were encountered were dealt with promptly. By following the
methodology of the client, phase one of the project is 95%, but there is still some work to
be done on phase two of the project before the database is complete.
Appendix A
Program Used to Read
import sys, os import re
IcedB = {}
FILE_EXT = ('CMP','DEN','FLX','FSC','MOD','THK','SHR','HIF')
regEx = re.compile('[:=]')
for root,dirs,files in os.walk(r'C:\Documents and Settings\DoyleC\Desktop\ICEDAT\icedat'):
for fileName in files:
ifo = file((root+r'/'+fileName), 'r')
ext = os.path.splitext(fileName)[1][-3:] if (ext in FILE_EXT): testTime = 0; direct = 0; Calib = 0; DATA = False AVG = False CALIB = False for line in ifo:
line = line.strip()
## ==== CMP =======
if DATA and ('CMP' == ext):
values = line.split()
if len(values) == 5:
testTime['Data'].append((values))
elif len(values) == 4:
testTime['Mean'] = values[0]
testTime['Std'] = values[1]
testTime['Num'] = values[3]
elif 'Test' in line:
DATA = False
## ==== DEN =======
elif DATA and ('DEN' == ext):
values = line.split()
if len(values) == 3:
if not AVG:
testTime['Data'].append((values))
else:
testTime['Mean'] = (values)
AVG = False
elif len(values) == 0:
AVG = True
elif 'Ice' in line:
DATA = False
## ==== FLX =======
elif DATA and ('FLX' == ext):
if len(values) == 5:
testTime['Data'].append((values,direct))
elif len(values) == 0:
direct = 'Up'
elif len(values) == 4:
testTime['Mean'] = values[0]
testTime['Std'] = values[1]
testTime['Num'] = values[3]
elif 'Time' in line:
DATA = False
## ==== FSC =======
elif DATA and ('FSC' == ext):
if ':' in line:
key, value = regEx.split(line) key = key.strip()
value = value.strip()
if 'Test time' in key:
time = value
if time not in IcedB[project][test][ext]: IcedB[project][test][ext][time] = {}
testTime = IcedB[project][test][ext][time]
elif 'Time' in key:
timeMain = value
elif 'Location' in key:
location = value
testTime[location] = {}
testTimeLoc = testTime[location]
testTimeLoc['Data'] = []
testTimeLoc['MainTime'] = timeMain
ifo.next() ifo.next() else: values = line.split()
if len(values) == 6:
testTimeLoc['Data'].append(values)
elif len(values) == 2:
testTimeLoc['Mean'] = values[1]
elif len(values) == 4:
testTimeLoc['Std'] = values[1]
testTimeLoc['Num'] = values[3]
## ==== THK =======
elif DATA and ('THK' == ext):
values = line.split()
if len(values) == 3:
if 'mean' not in line:
testTime['Data'].append(values)
else:
testTime['Mean'] = values[1:3]
elif len(values) == 4:
if 'std dev' in line:
testTime['Std'] = values[2:4]
elif '#' in line:
testTime['Samples'] = values[2:4]
## ==== MOD =======
elif DATA and ('MOD' == ext):
values = line.split()
if len(values) == 7:
testTime['Data'].append(values)
elif 'mean' in line:
testTime['Mean'] = values[1:3]
elif 's.d.' in line:
testTime['Std'] = values[1:3]
elif 'n=' in line:
testTime['Num'] = values[1]
## ==== HIF =======
elif DATA and ('HIF' == ext):
if ':' in line:
key, value = line.split(':')
if 'Plate' in key: plate = value.strip() if 'Traverse' in key: traverse = value.strip() if 'Time' in key: time = value.strip() if 'Estimated' in key: strength = value.strip() IcedB[project][test][ext]['Data'] = {}
IcedB[project][test][ext]['Data'][time] =
{} testTime =
IcedB[project][test][ext]['Data'][time]
testTime['Plate'] = plate
testTime['Traverse'] = traverse
testTime['Estimated'] = strength
testTime['Time'] = time
testTime['Data'] = []
else:
values = line.split()
if len(values) == 2:
testTime['Data'].append(values)
if 'Tangental load(N) =' in line:
first = line[:27]
second = line[29:]
first = first.split("= ",1)[1]
testTime['Slope'] = first.strip()
second = second.split("+ ",1)[1]
testTime['Offset'] = second.strip()
if 'Max. normal stress' in line:
testTime['MaxStress'] = line[23:]
if 'Min. normal stress' in line:
testTime['MinStress'] = line[24:]
DATA = False
## ==== CALIB ====
elif CALIB and ('HIF' == ext):
values = line.split()
testTime['Data'].append(values)
elif 'Tangental load(kg)' in line:
first = line[:28]
second = line[29:]
first = first.split("= ",1)[1]
testTime['CalSlope'] = first.strip()
second = second.split("+ ",1)[1]
testTime['CalOffset'] = second.strip()
CALIB = False
DATA = True
## ==== SHR =======
elif DATA and ('SHR' == ext):
values = line.split()
if len(values) == 4:
if 'n= ' not in line:
testTime['Data'].append(values)
else:
testTime['Mean'] = values[0]
testTime['Std'] = values[1]
testTime['Num'] = values[3]
DATA = False
## ===========
if not DATA and not CALIB and regEx.search(line): try:
key, value = regEx.split(line) key = key.strip()
value = value.strip() except:
if ('SHR' == ext):
if 'Calibration Factor' in line:
first = line[:36]
second = line[37:]
first = first.split(":",1)[1]
testTime['CalFactor'] = first.strip()
second = second.split(": ",1)[1]
testTime['PunchDiameter'] =
second.strip()
if 'Test Name' in key:
test = value
elif 'Project' in key:
project = value
if project not in IcedB: IcedB[project] = {}
if test not in IcedB[project]: IcedB[project][test] = {} IcedB[project][test][ext] = {}
elif 'Date' in key:
date = value
elif 'Location' in key:
testTime['Location'] = value
elif 'Test Machine' in key:
elif 'Calibration Factor' in key: factor = value
elif 'volume' in key:
testTime['Volume'] = value
elif 'force' in key:
testTime['Force'] = value
elif 'density' in key:
testTime['Density'] = value
elif 'Ice thickness' in key:
testTime['Ice thickness'] = value
elif 'Disc radius' in key:
testTime['Disc radius'] = value
elif 'Apparatus calibration' in key:
CALIB = True Calib = []
elif 'Tangental load(kg)' in key:
HIF = True Hif = []
elif 'Plate ID #' in key:
plate = value
elif 'Traverse speed' in key:
speed = value
elif 'Estimated ice strength' in key:
testTime['IceStrength'] = value
elif 'Time' in key or 'time' in key:
time = value
IcedB[project][test][ext][time] = {}
testTime = IcedB[project][test][ext][time]
if('CMP' == ext):
testTime['Test Machine'] = machine
testTime['Calibration Factor'] = factor
if ext in ('CMP', 'DEN', 'THK', 'MOD'):
testTime['Date'] = date
if ext in ('HIF'):
testTime['Hif'] = Hif
testTime['Date'] = date
testTime['Calib'] = Calib
testTime['Plate'] = plate
testTime['Speed'] = speed
if ext in ('FSC'):
DATA = True
if ('THK' == ext):
if 'center profile' in line:
testTime['Profile'] = line[:6]
if 'profil' in line:
testTime['Profile'] = line[:14]
if re.search('\#div|LENGTH|Position|#Div', line):
DATA = True
testTime['Data'] = []
if ('FLX' == ext):
direct = 'Down'
if re.search('Apparatus calibration', line):
IcedB[project][test][ext]['Calib'] = {}
testTime = IcedB[project][test][ext]['Calib']
testTime['Date'] = date
testTime['Data'] = [] ## =========== OUTPUT ============== ## ===================================== projects = IcedB.keys() projects.sort()
for project in projects: print project
tests = IcedB[project].keys() tests.sort()
for test in tests:
print ' ', test
parms = IcedB[project][test].keys() parms.sort()
for parm in parms:
times = IcedB[project][test][parm].keys() times.sort()
if parm == 'CMP':
for time in times:
c = IcedB[project][test]['CMP'][time]
print ' Comp' , c['Date'], time,
c['Location'], c['Test Machine'], c['Calibration
Factor']
for values in c['Data']:
print ' ', values
print ' ', c['Mean'], c['Std'], c['Num']
elif parm == 'DEN':
for time in times:
c = IcedB[project][test]['DEN'][time]
print ' Dens' , c['Date'], time,
c['Location'], c['Volume'], c['Force'],
c['Density']
for values in c['Data']:
print ' ', values
print ' m', c['Mean']
elif parm == 'FLX':
for time in times:
c = IcedB[project][test]['FLX'][time]
print ' FlxM', time , c['Location']
for values in c['Data']:
if values[1] == 'Up': print ' ', c['Mean'], c['Std'], c['Num'] print ' ', values elif parm == 'FSC':
for time in times:
c = IcedB[project][test]['FSC'][time]
keys.sort()
for location in keys: d = c[location]
print ' FlxS', time , location,
d['MainTime']
for values in d['Data']:
print ' ', values
print ' ',d['Mean'], d['Std'],
d['Num']
elif parm == 'MOD':
for time in times:
c = IcedB[project][test]['MOD'][time]
print ' MOD' , c['Date'], time,
c['Location'], c['Ice thickness'], c['Disc
radius']
for values in c['Data']:
print ' ', values
print ' m', c['Mean'], 'std', c['Std'],
'num', c['Num']
elif parm == 'THK':
for time in times:
c = IcedB[project][test]['THK'][time]
print ' THK' , c['Date'], time, c['Profile']
for values in c['Data']:
print ' ', values
print ' m', c['Mean'],'std',
c['Std'],'sample', c['Samples']
elif parm == 'HIF':
c = IcedB[project][test]['HIF']['Calib']
print ' HIF CALIB' , c['Date'],
c['CalSlope'], c['CalOffset']
for values in c['Data']:
print ' ', values
times = IcedB[project][test]['HIF']['Data'].keys()
times.sort()
for time in times:
c = IcedB[project][test]['HIF']['Data'][time]
print ' HIF DATA', time, c['Plate'],
c['Traverse'], c['Estimated'], c['Slope'],
c['Offset'], c['MaxStress'], c['MinStress']
for values in c['Data']:
print ' ', values
elif parm == 'SHR':
for time in times:
c = IcedB[project][test]['SHR'][time]
print ' SHR' , time, c['CalFactor'],
c['PunchDiameter']
for values in c['Data']:
print ' ', values
print ' ', c['Mean'], c['Std'], c['Num']
Appendix B
import re import sys import os IcedB = {} FILE_EXT = ('ISS') regEx = re.compile('[:=]')
for root,dirs,files in os.walk('N:/WEB/icedat'): for fileName in files:
ifo = file((root+r'/'+fileName), 'r')
ext = os.path.splitext(fileName)[1][-3:]
if (ext in FILE_EXT):
for line in ifo:
line = line.strip() ## ==== ISS ==== if ('ISS' == ext): if len(line) > 42: first = line[:40] second = line[41:] if ": " in first:
param1, first = first.split(": ",1)
elif "C " in first:
param1, first = first.split("C",1)
elif "at " in first:
param1, first = first.split("at",1)
else:
param1, first = first.split("s ",1)
if ") " in second:
param2, second = second.split(") ",1)
elif "C: " in second:
param2, second = second.split("C: ",1)
elif ": " in second:
param2, second = second.split(": ",1)
if 'Test Name' in param1:
test = first.strip()
if 'Project Number' in param2:
project = second.strip() if project not in IcedB:
IcedB[project] = {}
if test not in IcedB[project]: IcedB[project][test] = {} IcedB[project][test][ext] = {} issdB = IcedB[project][test][ext]
issdB['Seeding'] = {}
if 'Target ice thickness' in param1:
issdB['TargetIce'] = first.strip()
if 'EG/AD/S' in param2:
issdB['EG'] = second.strip()
issdB['TargetIceStrength'] = first.strip()
if 'Ice Type' in param2:
issdB['IceType'] = second.strip()
if 'Air temp.' in param1:
issdB['AirTemp'] = first.strip()
if 'Tank water' in param2:
issdB['TankWater'] = second.strip()
if 'Seeding completed' in param1:
issdB['SeedingDate'] = first.strip()
if 'Seed duration' in param2:
issdB['SeedDuration'] = second.strip()
if 'Seed volume' in param1:
first = first[2:]
issdB['SeedVolume'] = first.strip()
if 'Seed water' in param2:
second = second[2:]
issdB['SeedWater'] = second.strip()
if 'Target temp.' in param1:
first = first[2:]
issdB['TargetTemp'] = first.strip()
if 'Time to target' in param2:
issdB['Time2Target'] = second.strip()
if 'Avg temp. at' in param1:
first = first[2:]
issdB['AvgTempa'] = first.strip()
if 'Duration of plateau' in param2:
issdB['DurationOfP'] = second.strip()
if 'Avg temp. of' in param1:
issdB['AvgTempo'] = first.strip()
if 'Duration of freeze' in param2:
issdB['DurationOfF'] = second.strip()
if 'Total negative' in param1:
issdB['TotalNegative'] = first.strip()
if 'Thickness at end' in param2:
issdB['ThicknessAt'] = second.strip()
if 'Avg growth' in param1:
first = first[7:]
issdB['AvgGrowth'] = first.strip()
if 'Avg growth rate' in param2:
issdB['AvgGrowthRate'] =
second.strip()
if 'Warm-up' in param1:
issdB['WarmUp'] = first.strip()
if 'Length of' in param2:
issdB['LengthOf'] = second.strip()
if 'Time to tempering' in param1:
first = first[5:]
issdB['Time2Tempering'] =
first.strip()
if 'Avg tempering' in param2:
issdB['AvgTempering'] = second.strip()
if 'Final ice' in param1:
first = first[4:]
issdB['FinalIce'] = first.strip()
if 'Ice growth' in param2:
if 'Total growth' in param1:
first = first[7:]
issdB['TotalGrowth'] = first.strip()
if 'Total growth' in param2:
issdB['TotalGrowthRate'] = second.strip() elif ('(%)') in line: param1,first = line.split(") ",1) if 'tank(%' in param1:
issdB['Tank(%)'] = first.strip()
if 'room(%' in param1:
issdB['Room(%)'] = first.strip()
## =========== OUTPUT ============== ## ===================================== projects = IcedB.keys() projects.sort()
for project in projects: print project
tests = IcedB[project].keys() tests.sort()
for test in tests:
print ' ', test
parms = IcedB[project][test].keys() parms.sort()
for parm in parms:
times = IcedB[project][test][parm].keys()
times.sort()
#
if parm == 'ISS':
c = IcedB[project][test]['ISS']
print ' ISS','[',c['TargetIce'],',',c['EG'],']'
print ' ','[',c['TargetIceStrength'], ',',c['IceType'],']' print ' ','[',c['AirTemp'], ',',c['TankWater'],']' print ' ','[',c['SeedingDate'], ',',c['SeedDuration'],']' print ' ','[',c['SeedVolume'], ',',c['SeedWater'],']' print ' ','[',c['Tank(%)'], ',',c['Room(%)'],']' print ' ','[',c['TargetTemp'], ',',c['Time2Target'],']' print ' ','[',c['AvgTempa'], ',',c['DurationOfP'],']' print ' ','[',c['AvgTempo'], ',',c['DurationOfF'],']' print ' ','[',c['TotalNegative'], ',',c['ThicknessAt'],']' print ' ','[',c['AvgGrowth'], ',',c['AvgGrowthRate'],']'
print ' ','[',c['WarmUp'], ',',c['LengthOf'],']' print ' ','[',c['Time2Tempering'], ',',c['AvgTempering'],']' print ' ','[',c['FinalIce'], ',',c['IceGrowth'],']'
