Gwyddion – Free SPM (AFM, SNOM/NSOM, STM, MFM, …) data analysis software

file module utils

file module utils — Utility functions for file modules

Functions

gboolean	gwy_get_gboolean8 ()
gint16	gwy_get_gint16_le ()
gint16	gwy_get_gint16_be ()
guint16	gwy_get_guint16_le ()
guint16	gwy_get_guint16_be ()
gint32	gwy_get_gint32_le ()
gint32	gwy_get_gint32_be ()
guint32	gwy_get_guint32_le ()
guint32	gwy_get_guint32_be ()
gint64	gwy_get_gint64_le ()
gint64	gwy_get_gint64_be ()
guint64	gwy_get_guint64_le ()
guint64	gwy_get_guint64_be ()
gfloat	gwy_get_gfloat_le ()
gfloat	gwy_get_gfloat_be ()
gdouble	gwy_get_gdouble_le ()
gdouble	gwy_get_gdouble_be ()
gdouble	gwy_get_pascal_real_le ()
gdouble	gwy_get_pascal_real_be ()
gboolean	gwy_app_channel_check_nonsquare ()
gboolean	gwy_app_channel_title_fall_back ()
gboolean	gwy_app_xyz_title_fall_back ()
guint	gwy_app_channel_remove_bad_data ()
GwyDataField *	gwy_app_channel_mask_of_nans ()
GQuark	gwy_text_header_error_quark ()
GHashTable *	gwy_text_header_parse ()
const gchar *	gwy_text_header_context_get_section ()
guint	gwy_text_header_context_get_lineno ()
void	gwy_file_channel_import_log_add ()
void	gwy_file_volume_import_log_add ()
void	gwy_file_xyz_import_log_add ()
void	gwy_file_curve_map_import_log_add ()

Types and Values

#define	GWY_TEXT_HEADER_ERROR
enum	GwyTextHeaderError
	GwyTextHeaderParser
	GwyTextHeaderContext

Includes

#include <app/gwymoduleutils-file.h>

Description

Functions gwy_app_channel_check_nonsquare() and gwy_app_channel_title_fall_back() perform common tasks improving the imported of channels from foreign data files. Typically one calls them on all imported channel ids after storing the data fields the the container, if they are useful for a given file type.

The group of functions gwy_get_gint16_le(), gwy_get_gint16_be(), etc. is intended to portably read packed binary data structures that are commonly found in SPM files. They all work identically: the binary data value is read from the buffer, converted if necessary, and the provided buffer pointer is moved to point after the value to faciliate sequential reading.

As no buffer size is passed, obviously no buffer size checking is performed. The caller has to ensure the buffer is large enough -- it is expected the caller checks the total buffer size before starting to parse it.

For example to portably read the following packed struct stored in big-endian byte order:

struct {
    guint16 xres;
    guint16 yres;
    gfloat measure;
} header;

one can do (after checking the buffer size):

const guchar *p = buffer;
header.xres    = gwy_get_guint16_be(&p);
header.yres    = gwy_get_guint16_be(&p);
header.measure = gwy_get_gfloat_be(&p);

and p will point after measure in buffer after this snippet is finished.

The data types used in header do not matter (provided they are large enough to hold the values), the exact types are determined by the functions used. Therefore the reading would work identically if header was defined using common types:

struct {
    gint xres;
    gint yres;
    gdouble measure;
} header;

Functions

gwy_get_gboolean8 ()

gboolean
gwy_get_gboolean8 (const guchar **ppv);

Reads a boolean value stored as a signle byte from a binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to boolean (stored as a signle byte) in a memory buffer.

Returns

The gboolean value read from the buffer.

Since: 2.3

gwy_get_gint16_le ()

gint16
gwy_get_gint16_le (const guchar **ppv);

Reads a signed 16bit integer value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a little-endian signed 16bit integer in a memory buffer.

Returns

The gint16 value read from the buffer.

Since: 2.3

gwy_get_gint16_be ()

gint16
gwy_get_gint16_be (const guchar **ppv);

Reads a signed 16bit integer value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a big-endian signed 16bit integer in a memory buffer.

Returns

The gint16 value read from the buffer.

Since: 2.3

gwy_get_guint16_le ()

guint16
gwy_get_guint16_le (const guchar **ppv);

Reads an unsigned 16bit integer value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a little-endian unsigned 16bit integer in a memory buffer.

Returns

The guint16 value read from the buffer.

Since: 2.3

gwy_get_guint16_be ()

guint16
gwy_get_guint16_be (const guchar **ppv);

Reads an unsigned 16bit integer value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a big-endian unsigned 16bit integer in a memory buffer.

Returns

The guint16 value read from the buffer.

Since: 2.3

gwy_get_gint32_le ()

gint32
gwy_get_gint32_le (const guchar **ppv);

Reads a signed 32bit integer value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a little-endian signed 32bit integer in a memory buffer.

Returns

The gint32 value read from the buffer.

Since: 2.3

gwy_get_gint32_be ()

gint32
gwy_get_gint32_be (const guchar **ppv);

Reads a signed 32bit integer value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a big-endian signed 32bit integer in a memory buffer.

Returns

The gint32 value read from the buffer.

Since: 2.3

gwy_get_guint32_le ()

guint32
gwy_get_guint32_le (const guchar **ppv);

Reads an unsigned 32bit integer value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a little-endian unsigned 32bit integer in a memory buffer.

Returns

The guint32 value read from the buffer.

Since: 2.3

gwy_get_guint32_be ()

guint32
gwy_get_guint32_be (const guchar **ppv);

Reads an unsigned 32bit integer value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a big-endian unsigned 32bit integer in a memory buffer.

Returns

The guint32 value read from the buffer.

Since: 2.3

gwy_get_gint64_le ()

gint64
gwy_get_gint64_le (const guchar **ppv);

Reads a signed 64bit integer value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a little-endian signed 64bit integer in a memory buffer.

Returns

The gint64 value read from the buffer.

Since: 2.3

gwy_get_gint64_be ()

gint64
gwy_get_gint64_be (const guchar **ppv);

Reads a signed 64bit integer value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a big-endian signed 64bit integer in a memory buffer.

Returns

The gint64 value read from the buffer.

Since: 2.3

gwy_get_guint64_le ()

guint64
gwy_get_guint64_le (const guchar **ppv);

Reads an unsigned 64bit integer value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a little-endian unsigned 64bit integer in a memory buffer.

Returns

The guint64 value read from the buffer.

Since: 2.3

gwy_get_guint64_be ()

guint64
gwy_get_guint64_be (const guchar **ppv);

Reads an unsigned 64bit integer value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a big-endian unsigned 64bit integer in a memory buffer.

Returns

The guint64 value read from the buffer.

Since: 2.3

gwy_get_gfloat_le ()

gfloat
gwy_get_gfloat_le (const guchar **ppv);

Reads a single-precision IEEE float value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a little-endian single-precision IEEE float in a memory buffer.

Returns

The gfloat value read from the buffer.

Since: 2.3

gwy_get_gfloat_be ()

gfloat
gwy_get_gfloat_be (const guchar **ppv);

Reads a single-precision IEEE float value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a big-endian single-precision IEEE float in a memory buffer.

Returns

The gfloat value read from the buffer.

Since: 2.3

gwy_get_gdouble_le ()

gdouble
gwy_get_gdouble_le (const guchar **ppv);

Reads a double-precision IEEE float value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a little-endian double-precision IEEE float in a memory buffer.

Returns

The gdouble value read from the buffer.

Since: 2.3

gwy_get_gdouble_be ()

gdouble
gwy_get_gdouble_be (const guchar **ppv);

Reads a double-precision IEEE float value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a big-endian double-precision IEEE float in a memory buffer.

Returns

The gdouble value read from the buffer.

Since: 2.3

gwy_get_pascal_real_le ()

gdouble
gwy_get_pascal_real_le (const guchar **ppv);

Reads a six-byte Pascale Real value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a little-endian six-byte Pascal Real in a memory buffer.

Returns

The floating point value read from the buffer as a gdouble.

Since: 2.3

gwy_get_pascal_real_be ()

gdouble
gwy_get_pascal_real_be (const guchar **ppv);

Reads a six-byte Pascale Real value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

Parameters

ppv

Pointer to a pointer to a big-endian six-byte Pascal Real in a memory buffer.

Returns

The floating point value read from the buffer as a gdouble.

Since: 2.3

gwy_app_channel_check_nonsquare ()

gboolean
gwy_app_channel_check_nonsquare (GwyContainer *data,
                                 gint id);

Sets `realsquare' for a channel highly non-square pixels.

The threshold for highly non-square is somewhat arbitrary. Fortunately, most files encoutered in practice have the measure ratio either very close to 1, larger or equal than 2.

Parameters

data	A data container.
id	Data channel id.

Returns

TRUE if the channel was found to have highly non-square pixels and `realsquare' was set (otherwise it was unset).

Since: 2.3

gwy_app_channel_title_fall_back ()

gboolean
gwy_app_channel_title_fall_back (GwyContainer *data,
                                 gint id);

Adds a channel title based on data field units.

The guess is very simple, but probably better than `Unknown channel' in most cases. If there already is a title it is left intact, making use of this function as a fall-back easier.

Parameters

data	A data container.
id	Data channel id.

Returns

TRUE if the title was set (either by this function or before).

Since: 2.3

gwy_app_xyz_title_fall_back ()

gboolean
gwy_app_xyz_title_fall_back (GwyContainer *data,
                             gint id);

Adds a XYZ data title based on surface value units.

The guess is very simple, but probably better than `Unknown channel' in most cases. If there already is a title it is left intact, making use of this function as a fall-back easier.

Parameters

data	A data container.
id	XYZ surface data id.

Returns

TRUE if the title was set (either by this function or before).

Since: 2.45

gwy_app_channel_remove_bad_data ()

guint
gwy_app_channel_remove_bad_data (GwyDataField *dfield,
                                 GwyDataField *mfield);

Replaces bad data points with some neutral values.

Since Gwyddion has no concept of bad data points, they are usually marked with a mask and replaced with some neutral values upon import, leaving the user to decide how to proceed further. This helper function performs such replacement.

Parameters

dfield	A data field. The values of bad data points are ignored and might be even left uninitialized.
mfield	A mask field containing 1.0 in place of good data points, 0.0 in place of bad points. It will be inverted to become the mask of bad points.

Returns

The number of bad data points replaced. If zero is returned, all points are good and there is no need for masking.

Since: 2.14

gwy_app_channel_mask_of_nans ()

GwyDataField *
gwy_app_channel_mask_of_nans (GwyDataField *dfield,
                              gboolean removebad);

Creates a mask corresponding to not-a-number and infinite values in a data field.

NaNs and infinities are not normally allowed in Gwyddion so this function should be used upon import if the source file format can represent such values.

The invalid values can be automatically removed using the same method as gwy_app_channel_remove_bad_data().

Parameters

dfield	A data field possibly containing NaNs and infinities.
removebad	Automatically remove the invalid values.

Returns

A newly created data field with 1.0 in pixels corresponding to a NaN or infinity in dfield . If no such pixels are present, NULL is returned.

Since: 2.38

gwy_text_header_error_quark ()

GQuark
gwy_text_header_error_quark (void);

Returns error domain for expression parsin and evaluation.

See and use GWY_TEXT_HEADER_ERROR.

Returns

The error domain.

Since: 2.18

gwy_text_header_parse ()

GHashTable *
gwy_text_header_parse (gchar *header,
                       const GwyTextHeaderParser *parser,
                       gpointer user_data,
                       GError **error);

Parses a line-oriented text header into a hash table.

See GwyTextHeaderParser for details of memory and error handling.

Lines consisting only of whitespace are ignored.

Parameters

header	Text header to parse. It must be `NULL`-terminated and writable. Its contents will be modified to directly embed the hash keys and/or values. It must not be freed while the returned hash table is in use.
parser	Parser specification.
user_data	User data passed to parser callbacks.
error	Error to set on fatal errors.

Returns

A newly created hash table with values indexed by they keys found in the header.

Since: 2.18

gwy_text_header_context_get_section ()

const gchar *
gwy_text_header_context_get_section (const GwyTextHeaderContext *context);

Gets the currently open section.

This function may be called if no sectioning is defined. It simply returns NULL then.

Parameters

context

Header parsing context.

Returns

The name of the currently open section, NULL if there is none.

Since: 2.18

gwy_text_header_context_get_lineno ()

guint
gwy_text_header_context_get_lineno (const GwyTextHeaderContext *context);

Gets the current header line.

Parameters

context

Header parsing context.

Returns

The current line number, starting from zero.

Since: 2.18

gwy_file_channel_import_log_add ()

void
gwy_file_channel_import_log_add (GwyContainer *data,
                                 gint id,
                                 const gchar *filetype,
                                 const gchar *filename);

Logs the import of a channel from third-party file.

This is a convenience wrapper for gwy_app_channel_log_add(). The source id will be set to -1. The file name will be added to function arguments.

Parameters

data	A data container.
id	Data channel id.
filetype	File type, i.e. the name of the function importing the data (without any "file::" prefix). Since 2.38 it is possible to pass `NULL` to fill the name of the currently running file type function automatically.
filename	Name of the imported file. If it is not valid UTF-8, it will be converted to UTF-8 using `g_filename_to_utf8()`. Failing even that, non-ASCII characters will be escaped.

Since: 2.35

gwy_file_volume_import_log_add ()

void
gwy_file_volume_import_log_add (GwyContainer *data,
                                gint id,
                                const gchar *filetype,
                                const gchar *filename);

Logs the import of volume data from third-party file.

This is a convenience wrapper for gwy_app_volume_log_add(). The source id will be set to -1. The file name will be added to function arguments.

Parameters

data	A data container.
id	Volume data id.
filetype	File type, i.e. the name of the function importing the data (without any "file::" prefix). Since 2.38 it is possible to pass `NULL` to fill the name of the currently running file type function automatically.
filename	Name of the imported file. If it is not valid UTF-8, it will be converted to UTF-8 using `g_filename_to_utf8()`. Failing even that, non-ASCII characters will be escaped.

Since: 2.35

gwy_file_xyz_import_log_add ()

void
gwy_file_xyz_import_log_add (GwyContainer *data,
                             gint id,
                             const gchar *filetype,
                             const gchar *filename);

Logs the import of xyz data from third-party file.

This is a convenience wrapper for gwy_app_xyz_log_add(). The source id will be set to -1. The file name will be added to function arguments.

Parameters

data	A data container.
id	XYZ surface data id.
filetype	File type, i.e. the name of the function importing the data (without any "file::" prefix). It is possible to pass `NULL` to fill the name of the currently running file type function automatically.
filename	Name of the imported file. If it is not valid UTF-8, it will be converted to UTF-8 using `g_filename_to_utf8()`. Failing even that, non-ASCII characters will be escaped.

Since: 2.45

gwy_file_curve_map_import_log_add ()

void
gwy_file_curve_map_import_log_add (GwyContainer *data,
                                   gint id,
                                   const gchar *filetype,
                                   const gchar *filename);

Logs the import of volume data from third-party file.

This is a convenience wrapper for gwy_app_curve_map_log_add(). The source id will be set to -1. The file name will be added to function arguments.

Parameters

data	A data container.
id	Volume data id.
filetype	File type, i.e. the name of the function importing the data (without any "file::" prefix). Since 2.38 it is possible to pass `NULL` to fill the name of the currently running file type function automatically.
filename	Name of the imported file. If it is not valid UTF-8, it will be converted to UTF-8 using `g_filename_to_utf8()`. Failing even that, non-ASCII characters will be escaped.

Since: 2.60

Types and Values

GWY_TEXT_HEADER_ERROR

#define GWY_TEXT_HEADER_ERROR gwy_text_header_error_quark()

Error domain for text header parsing. Errors in this domain will be from the GwyTextHeaderError enumeration. See GError for information on error domains.

Since: 2.18

enum GwyTextHeaderError

Error codes returned by text header parsing.

Some errors, in particular GWY_TEXT_HEADER_ERROR_KEY and GWY_TEXT_HEADER_ERROR_VALUE are expected to be raised by user callbacks (they are not restricted to these codes though).

Members

GWY_TEXT_HEADER_ERROR_SECTION_NAME	Section name is invalid. It is raised by the parser only for an empty section name.
GWY_TEXT_HEADER_ERROR_SECTION_END	Section ended when a different section or no section was open. Note that `gwy_text_header_context_get_section()` returns the section being closed at the time this error is raised.
GWY_TEXT_HEADER_ERROR_SECTION_START	Section started before the previous ended. This is raised only if `endsection_template` is set.
GWY_TEXT_HEADER_ERROR_PREFIX	Line lacks the mandatory prefix.
GWY_TEXT_HEADER_ERROR_GARBAGE	Line cannot be parsed into a key-value pair.
GWY_TEXT_HEADER_ERROR_KEY	Key name is invalid, namely empty.
GWY_TEXT_HEADER_ERROR_VALUE	Value is invalid. This is never raised by the parser.
GWY_TEXT_HEADER_ERROR_TERMINATOR	The text header has ended without being terminated by specified terminator.

Since: 2.18

GwyTextHeaderParser

typedef struct {
    const gchar *comment_prefix;
    const gchar *section_template;
    const gchar *endsection_template;
    const gchar *section_accessor;
    const gchar *line_prefix;
    const gchar *key_value_separator;
    const gchar *terminator;
    gboolean (*item)(const GwyTextHeaderContext *context,
                     GHashTable *hash,
                     gchar *key,
                     gchar *value,
                     gpointer user_data,
                     GError **error);
    gboolean (*section)(const GwyTextHeaderContext *context,
                        const gchar *name,
                        gpointer user_data,
                        GError **error);
    gboolean (*endsection)(const GwyTextHeaderContext *context,
                           const gchar *name,
                           gpointer user_data,
                           GError **error);
    void (*end)(const GwyTextHeaderContext *context,
                gsize length,
                gpointer user_data);
    gboolean (*error)(const GwyTextHeaderContext *context,
                      GError *error,
                      gpointer user_data);
    GDestroyNotify destroy_key;
    GDestroyNotify destroy_value;
} GwyTextHeaderParser;

Text header parser specification.

Memory considerations: In general, the parser attempts to reuse the contents of header directly for the hash keys and values. There are two cases when it cannot: sectioning implies that keys must be constructed dynamically and the use of item callback means the parser has no control on what is inserted into the hash table.

This means that the item callback must free key if sectioning is used and it is not going to actually use it as the hash table key. And, of course, suitable destroy_key and destroy_value functions must be set.

Members

const gchar *`comment_prefix`;	Prefix denoting comment lines. It is possible to specify multiple prefixes by separating them with newline ("\n").
const gchar *`section_template`;	Section start template. It must contain the character "\x1a" in the place where the section name apprears. For example, "[Section \x1a]".
const gchar *`endsection_template`;	Section end template. It may or may not contain the substitute character "\x1a" depending on whether the section end markers contain the section name. It is invalid to set
const gchar *`section_accessor`;	Glue to put between the section name and key when constructing hash table keys. It is invalid to set `section_accessor` without setting `section_template` . Typically, "::" is used.
const gchar *`line_prefix`;	Mandatory prefix of each line. Note this applies only to key-value lines. The templates, such comments, sections and terminators, must still include `line_prefix` if they start with it.
const gchar *`key_value_separator`;	The string separating keys and values on each line. Typically, "=" or ":" is used. When left `NULL`, whitespace plays the role of the separator. Of course, keys cannot contain whitespace then.
const gchar *`terminator`;	Line that marks end of the header. It is mandatory if specified, `GWY_TEXT_HEADER_ERROR_TERMINATOR` is raised when the header ends sooner than `terminator` is found.
`item` ()	Callback called when a key-value pair is found. If set it is responsible for inserting the pair to the hash table with `g_hash_table_replace()`. It is free to insert a different pair or nothing. It must return `FALSE` if it raises an error.
`section` ()	Callback called when a section starts. It must return `FALSE` if it raises an error.
`endsection` ()	Callback called when a section end. It must return `FALSE` if it raises an error.
`end` ()	Callback called when the parsing finishes successfully, `length` contains the length of parsed text in bytes.
`error` ()	Callback called when an error occurs, including errors raised by other user callbacks. If it returns `TRUE` the error is considered fatal and the parsing terminates immediately. If it is left unset no errors are fatal hence no errors are reported to the caller.
GDestroyNotify `destroy_key`;	Function to destroy keys, passed to `g_hash_table_new_full()`. It is invalid to set `destroy_key` if `item` callback is not set.
GDestroyNotify `destroy_value`;	Function to destroy values, passed to `g_hash_table_new_full()`. It is invalid to set `destroy_value` if `item` callback is not set.

Since: 2.18

GwyTextHeaderContext

typedef struct _GwyTextHeaderContext GwyTextHeaderContext;

GwyTextHeaderContext represents the parsing state. It is an opaque data structure and should be only manipulated with the functions below.

Since: 2.18