Imptools Documentation

This page contains the Imptools Module documentation.

The imptools Module

This program implements a database importer that reads from ascii-input files to the django database. It’s generic and is controlled from a mapping file.

class imptools.rewrite.MappingFile(filepath, headblocks, commentchar, blockoffset, blockstep, errblock, startblock=None, endblock='n')

Bases: object

This class implements an object that represents an open file from which one can read blocks. The object keeps track of its own block-step speed and will return lines as defined by this speed. E.g. for a line-step speed of 0.5, it will return the same line twice in a row whereas for a step speed of 2, will return every second block etc.

If endblock is \n (default), the block will infact represent a line.

block_generator(fileobj, startblock=None, endblock='\n')

generator, stepping through blocks

readblock()

Return a block from the file.

This method understand both slower block stepping (0 < blockstep < 1) and faster (> 1)

imptools.rewrite.ftime(t0, t1)

formats time to nice format.

imptools.rewrite.get_value(linedata, column_dict)

Process one line/block of data. Linedata is a tuple that always starts with the raw string for the line. The function with its arguments is read from the column_dict and applied to the linedata. The result is returned, after checking for the NULL value.

imptools.rewrite.is_iter(iterable)

imptools.rewrite.log_trace(e, info='')

Intended to be called from inside a traceback exception with the exception object as first argument. Captures the latest traceback.

imptools.rewrite.make_outfile(file_dict, global_debug=False)

Process one file definition from a config dictionary by processing the file name stored in it and parse it according to the mapping.

file_dict - config dictionary representing one input file structure (for an example see e.g. mapping_vald3.py)

a function raising a RuntimeError exception will skip the current line being parsed.

imptools.rewrite.parse_mapping(mapping, debug=False)

Step through a list of mappings describing the relation between (usually ascii-)files and django database fields. This should ideally not have to be changed for different database types.

imptools.rewrite.read_mapping(fname)

Read the config dictionary from a file. Note: very unsafe, since the content gets executed. Have a look at createcfg() to see how it should look like.

imptools.rewrite.validate_mapping(mapping)

Check the mapping definition to make sure it contains a structure suitable for the parser.