Dump Format

Dump Format — Dumb dump file format used for data exchange between plug-in proxy and plug-ins

Overall Structure

The dump format consists of text lines, except for the actual data samples which are for efficiency reasons stored as arrays of (binary) IEEE double precision numbers.

Since dump files contain binary data (along with text lines) they have to be always opened in binary mode to avoid end-of-line conversions (on systems distinguishing between text and binary files). Unix end-of-lines are always used, i.e., lines are always terminated by a single LF (line feed) character, ASCII 0x0a. Plugin-proxy always writes data with Unix end-of-lines and although it may read dumps with CRLF end-of-lines without problems too, don't count on it.

String Data

The text lines have the form key=value. The keys are keys in data container created for the data, and the values are corresponding values. Except for a few special keys specified below, all values are stored as strings.

There are only a few types of data you may expect to find in or want to write to a dump file:

  • Data fields. They use some specific keys and you can read more about them below.
  • Metadata about the data. They start with /meta/ followed by an arbitrary key string. Not used directly by Gwyddion, but will appear in Metadata Browser.
  • Queer stuff (everything else). It's best to ignore it.

Data Fields

The data samples themselves are stored as a sequence of binary IEEE floating point numbers in little-endian byte order. The array is preceded by /0/data=[, a single LF character, and a single left bracket [. The data samples are closed by a sequence of two right brackets ]] and a single LF character.

The resolutions and physical dimensions are specified using special keys /0/data/xres, /0/data/yres, /0/data/xreal, and /0/data/yreal. Since the resolutions must be obviously known before data samples can be read, the lines /0/data/xres=x-resolution and /0/data/yres=y-resolution must appear somewhere before it in the file. It is also strongly recommended to specify physical dimensions too, though it is not strictly necessary if you don't mind some bogus default value (like 1 meter) is substituted then.

All the values are in base SI units, i.e., dimensions are in meters (not nanometres or kilometres), currents in ampers (not pikoampers), etc. The base units can be specified using the /0/data/unit-xy (lateral) and /0/data/unit-z (value, height) keys. If not specified, meters are assumed.

So a dump file could look (excluding the binary data):

/0/data/xres=240
/0/data/yres=240
/0/data/xreal=4.1e-08
/0/data/yreal=4.1e-08
/0/data/unit-xy=m
/0/data/unit-z=m
/0/data=[
[...
...
..]]

If you want to pass mask (or presentation) instead of the main data, just replace the /0/data with /0/mask (or /0/show) in the previous description. If they are present in the data, they are always written to the dump, so you don't need anything special to read them.

Reference Implementations

A few sample dump format implementations are included in Gwyddion source distribution.

C/C++.  The plug-in proxy itself is always the most complete reference C/C++ implementation. See namely text_dump_export() and text_dump_import() functions in modules/plugin-proxy.c.

Perl.  A sample Perl data-processing plug-in is included in the source distribution: plugins/process/cutoff.pl.

Python.  A sample Python data-processing plug-in is included in the source distribution: plugins/process/divide_by_2.py.