Commit e6988a71 authored by Istvan N Huszar's avatar Istvan N Huszar
Browse files

Partial bugfix for TIRLFile object ID redundancy problem.

parents
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# _______ _____ _____ _
# |__ __| |_ _| | __ \ | |
# | | | | | |__) | | |
# | | | | | _ / | |
# | | _| |_ | | \ \ | |____
# |_| |_____| |_| \_\ |______|
#
# Copyright (C) 2018-2020 University of Oxford
# Part of the FMRIB Software Library (FSL)
# Author: Istvan N. Huszar
# SHBASECOPYRIGHT
# DESCRIPTION
"""
Tensor Image Registration Library (TIRL) is an open-source general-purpose
image registration platform that integrates into the FMRIB Software Library
(FSL) and allows rapid prototyping of bespoke image registration pipelines.
TIRL was originally developed for registering sparsely sampled small
histological sections to whole-brain post-mortem MRI. However the variety
and customisability of the components should make TIRL suitable for a wider
range of applications.
If you use TIRL in your research, please cite:
IN Huszar, M Pallebage-Gamarallage, S Foxley, BC Tendler, A Leonte, M Hiemstra,
J Mollink, A Smart, S Bangerter-Christensen, H Brooks, MR Turner, O Ansorge,
KL Miller, M Jenkinson; Tensor Image Registration Library: Automated Non-Linear
Registration of Sparsely Sampled Histological Specimens to Post-Mortem MRI of
the Whole Human Brain; bioRxiv 849570; doi: https://doi.org/10.1101/849570
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #
# Overview of the main components of TIRL #
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #
TIRLObject : the base class for all objects in TIRL
---------------------------------------------------
The base class implements an instance counting feature and manages the saving
and loading of TIRL objects into/from TIRL files.
Domain : a collection of points in voxel and physical space
-----------------------------------------------------------
Represents a regular grid ("compact domain") of a certain shape or a sparse
point set ("non-compact domain"). Every point in the Domain is characterised by
its integer coordinates denoting its position in voxel space. Domain defines a
list of Transformation objects (altogether referred to as the "transformation
chain") that map voxel coordinates into real-world space ("physical
coordinates"). The transformation chain is an extension of the qform/sform
concept defined by the NIfTI-1 standard, allowing images to have intuitive
mapping into physical space, which may also include non-linear transformations.
In FSL, transforming images from one "space" to another is achieved by applying
a previously estimated FLIRT matrix or FNIRT field to a NIfTI image, given a
reference image. In TIRL, the different spaces are represented by different
Domain instances, and transforming an image to another space is achieved by
"evaluating" the image on another Domain. As all images are defined on a
domain, the aim of image registration in TIRL terminology is to find
the transformation chain that best maps the points of one image onto the domain
of the other image.
Any Domain object can be saved as or loaded from a TIRL file (with the default
extension .dom). Domain objects normally get saved as part of either TField or
TImage objects (see these below).
Transformation : coordinate transformation
------------------------------------------
Object that maps input coordinates to output coordinates according to its
internal mapping function. Each subclass of Transformation implements a
specific mapping rule on the basis of transformation parameters and
metaparameters. Metaparameters are expected to be set at construction time
and remain static for the lifetime of the Transformation. Metaparameters should
modulate how the Transformation works, but should not be involved in the
mapping. Parameters are expected to be directly involved in the mapping and
change during optimisation. The two major subclasses of Transformation are
TxLinear for linear transformations and TxNonLinear for non-linear
transformations. Any Transformation object can be saved as or loaded from a
TIRL file (with the default extension .tx). Transformations are normally saved
as part of Domain, TField, or TImage objects.
TransformationGroup : concatenated transformations
--------------------------------------------------
A subclass of Transformation that allows several separate Transformations be
concatenated to a single Transformation object. The member transformations
still exist after the concatenation, and their parameters can be manipulated
either directly or by the TransformationGroup.
ParameterVector : handles transformation parameters
---------------------------------------------------
Stores and manages the parameters of all Transformation objects. Allows the
definition of lower and upper bounds for each parameter. Allows to lock any
subset of the parameters, so that their values can be kept constant. Parameters
are automatically locked if the lower and upper bounds become equal. Instead of
directly changing the parameters, Optimisers interact with this class via the
get(), set() and update() methods to manipulate the non-locked set of
parameters, i.e. retrieve them, assign new values to them, or increment their
existing values. Whenever a parameter is changed, the signature of the
ParameterVector changes, so that updates can be tracked. This feature is used
to cache large deformation fields that do not need to be recomputed unless the
until the transformation parameters change.
Interpolator : customised interpolation
---------------------------------------
Implements an interpolation routine, which is then used to resample data from
discrete data points of images and fields at continuous coordinates. TIRL uses
the interpolators of the Scipy.interpolation library by default. An example
(the FSLInterpolator) is provided how a custom-defined interpolator
(implemented in C++) can be wrapped as an Interpolator and used in TIRL. The
FSLInterpolator implements trilinear interpolation, which is the default in FSL.
TField : tensor field
---------------------
Implements a rank-L tensor field that is defined on a continuous N-dimensional
(sparse or compact) Domain. Discrete data are stored in an ndarray or memmap
(depending on user specification and the availability of RAM) in either
tensor-major or voxel-major layout. For continuous grid points data are
interpolated using the customisable Interpolator object specified at the
construction of TField. TField supports arithmetic operations with other TField
instances and ndarrays; these operations are carried out on the voxel array.
TField further supports ndarray-style slicing of the voxel array: convex slices
remain TField objects, in which every voxel maps to the same physical location
as before the slicing. TField supports evaluation on other domains by
interpolation, which makes it convenient to downsample or upsample the field on
Domains with fewer and more grid points, respectively. Tensor components can be
visualised via the TIRLVision library, calling TField.preview(). A TField can
be saved into and loaded from a TIRL file (with the default .tf extension).
TImage : tensor image
---------------------
A subclass of TField with additional methods tailored to image handling. TImage
readily imports data from many image file formats, including TIFF, PNG, JPEG,
BMP, GIF, PGM, NIFTI, SVS and others. Furthermore, a TImage can be constructed
from a TField or an array-like object. TImage generalises the image concept
such that every image is a set of L-rank tensors defined on a continuous
N-dimensional Domain. In addition to TField TImage implements the
ResolutionManager, which preserves the high-resolution copy of the input data
after subsampling the TImage on a sparser grid. This allows switching back and
forth between different resolution levels without degradation of data quality.
This feature also simplifies the syntax of writing multi-resolution
registration scripts. As the resampling does not create a new TImage instance,
transformations do not get de-referenced from the image, and their optimisation
can be continued at the new resolution level. In addition to TField, TImage
defines a mask that can be used to change the relative contribution of certain
parts of the image to the final alignment. This is normally used to exclude
areas where the source and the target images are different, e.g. because of
some artefact of pathology. Finally, TImage just like TField can be saved as or
read from a TIRL file (with the default extension .timg).
Operation : calculations on large tensor fields/images
------------------------------------------------------
Most universal functions can be readily applied to TField and TImage, implying
that both the input and the result of the operation is stored in memory. This
(e.g. applying a smoothing kernel) may not be possible for excessively large
images, such as high-resolution histology images. The Operation package
provides a programmable interface to perform custom operations on TField/TImage
in a chunkwise fashion. TensorOperator preserves the tensor shape and flattens
the spatial domain of the tensor field, such that an operation can be carried
out on the tensors, and get vectorised along the voxels. SpatialOperator also
preserves the tensor shape but extracts overlapping tiles from the tensor
field, allowing to perform calculations that depend on the voxel neighbourhood.
Tile sizes and shapes are optimised for speed, and overlapping regions prevent
boundary-errors on kernel operations.
Cost : measures of image dissimilarity
--------------------------------------
Subclasses of the Cost base function implement different models to quantify the
dissimilarity of two TImage instances. Depending on the mathematical nature of
the model, some Cost objects can also return cost gradients for gradient-based
optimisation. To initialise a Cost object, one must specify the "source" and
the "target" TImages. TIRL provides subclasses for the following dissimilarity
models: sum-of-squared intensities, normalised mutual information and
Modality-Independent Neighbourhood Descriptor (MIND) (see Heinrich et al, 2012,
Medical Image Analysis). Users of TIRL are encouraged to extend the repertoire
of cost functions by implementing their own models of image dissimilarity.
Regulariser : measures of transformation consistency
----------------------------------------------------
Subclasses of the Regulariser base class implement the calculation of specific
penalty scores based on the parameters of a Transformation or
TransformationGroup. These normally aim to repress unwanted directions in the
optimisation that would lead to physically implausible or inconsistent
transformations. To initialise a Regulariser, one must specify a Transformation
or TransformationGroup instance. TIRL provides regularisers for calculating
Euclidean norms of the transformation parameters, membrane energy, and
diffusion regularisation.
Optimiser : the motor of image registration
-------------------------------------------
Calling an initialised Optimiser object starts the image registration. To set
up an Optimiser object, one needs to specify the Transformation or
TransformationGroup, the Cost and Regulariser objects, and the stopping
criteria. For any given optimisation, multiple Cost and Regulariser objects can
be specified. As the Optimiser runs, it computes the total registration cost by
summing the contributions of all Cost and Regulariser objects and repeatedly
updates the parameters of the Transformation or TransformationGroup until one
of the stopping conditions is reached. At this point the cost is hopefully
minimal, corresponding to a good alignment between the source and target images.
Subclasses of the Optimiser base class implement wrappers around various
optimisation algorithms available in the SciPy.optimize and NLOpt libraries.
Users of TIRL are encouraged to extend this repertoire by implementing their
own optimisation strategies.
"""
# DEPENDENCIES
# PACKAGE IMPORTS
# from tirl import fsl
# from tirl import settings as ts
# if ts.ENABLE_VISUALISATION:
# from tirl import tirlvision
# DEFINITIONS
__author__ = "Istvan N. Huszar"
__copyright__ = "Copyright (C) 2018-2020 University of Oxford"
__credits__ = ["Mark Jenkinson"]
__license__ = "FSL"
__version__ = "2.2.1"
__maintainer__ = "Istvan N Huszar"
__status__ = "active"
# IMPLEMENTATION
def home(*args):
"""
Returns the home directory of TIRL (if no arguments are specified).
Arguments are interpreted as elements of a relative path from TIRL's home
directory. In this case, the full absolute path to the specified object
will be returned.
"""
import os
homedir = os.path.dirname(__file__)
if args:
return os.path.join(homedir, *args)
else:
return homedir
def set_sharedir(path):
"""
Set the location of TIRL's "share" directory.
"""
import os
import re
from glob import glob
with open(home("settings.py"), "r+") as f:
content = f.read()
pattern = r"TIRLSHARE\s*=\s*.+(?=\n)"
try:
path = os.path.abspath(glob(path)[0])
except:
pass
replacement = f"TIRLSHARE = \"{os.path.abspath(path)}\""
content = re.sub(pattern, replacement, content)
with open(home("settings.py"), "w") as f:
f.write(content)
def sharedir(*args):
import os
import tirl.settings as ts
d = ts.TIRLSHARE
if args:
d = os.path.join(d, *args)
return os.path.realpath(d)
def set_testdir(path):
"""
Set the location of TIRL's "tests" directory.
"""
import os
import re
from glob import glob
with open(home("settings.py"), "r+") as f:
content = f.read()
pattern = r"TIRLTESTS\s*=\s*.+(?=\n)"
try:
path = os.path.abspath(glob(path)[0])
except:
pass
replacement = f"TIRLTESTS = \"{os.path.abspath(path)}\""
content = re.sub(pattern, replacement, content)
with open(home("settings.py"), "w") as f:
f.write(content)
def testdir(*args):
import os
import tirl.settings as ts
d = ts.TIRLTESTS
if args:
d = os.path.join(d, *args)
return os.path.realpath(d)
def testimg():
""" Returns a test TImage. """
from tirl.timage import TImage
impath = sharedir("resources", "testimage", "testimg.png")
img = TImage.fromfile(impath, tensor_axes=(2,))
return img
def load(fname):
"""
Loads a TIRL object from file.
:param fname: file path
:type fname: str
:returns: TIRLObject of the respective kind
:rtype: TIRLObject
"""
from tirl.tirlobject import TIRLObject
return TIRLObject.load(fname)
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# _______ _____ _____ _
# |__ __| |_ _| | __ \ | |
# | | | | | |__) | | |
# | | | | | _ / | |
# | | _| |_ | | \ \ | |____
# |_| |_____| |_| \_\ |______|
#
# Copyright (C) 2018-2020 University of Oxford
# Part of the FMRIB Software Library (FSL)
# Author: Istvan N. Huszar
# SHBASECOPYRIGHT
# DESCRIPTION
""" Beta function and beta class indicators. """
# DEPENDENCIES
import tirl
from tirl.tirlobject import InstanceCounterMeta
# DEFINITIONS
__version__ = tirl.__version__
# IMPLEMENTATION
def beta_function(func):
"""
Decorator that indicates when a certain function/method is untested. Warns
the user that results from the respective function/method might be
incorrect.
:param func: untested function or method
:type func: Callable
:returns: untested function or method with a warning message
:rtype: Callable
"""
import warnings
from functools import wraps
@wraps(func)
def wrapped(*args, **kwargs):
warnings.warn(
f"{func.__name__} is a beta function in TIRL {__version__}. "
f"Use with caution: results might be incorrect.")
return func(*args, **kwargs)
return wrapped
class BetaClassMeta(InstanceCounterMeta):
"""
Metaclass that indicates when a certain class is untested. Warns the user
that results from the respective class might be incorrect.
"""
def __new__(cls, name, bases, attrs):
import warnings
from functools import wraps
initfunc = attrs['__init__']
@wraps(initfunc)
def wrapped(*args, **kwargs):
self = args[0]
warnings.warn(f"{self.__class__.__name__} is a beta class in TIRL "
f"{__version__}. Use with caution: results might be incorrect.")
initfunc(*args, **kwargs)
attrs['__init__'] = wrapped
return super(BetaClassMeta, cls).__new__(cls, name, bases, attrs)
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# _______ _____ _____ _
# |__ __| |_ _| | __ \ | |
# | | | | | |__) | | |
# | | | | | _ / | |
# | | _| |_ | | \ \ | |____
# |_| |_____| |_| \_\ |______|
#
# Copyright (C) 2018-2020 University of Oxford
# Part of the FMRIB Software Library (FSL)
# Author: Istvan N. Huszar
# SHBASECOPYRIGHT
# DEPENDENCIES
import os
import shutil
import warnings
import tempfile
import numpy as np
# TIRL IMPORTS
from tirl import settings as ts
class Buffer(object):
"""
Buffer class. Uniform interface for memory-mapped and in-memory arrays.
Automatically deletes linked files on the hard disk when discarded. This
class is a successor of the Buffer namedtuple that was used in the original
implementation of Domain.
"""
def __init__(self, arr=None, fname=None, file_no=None, signature=None):
# Signature
self.signature = signature
# File ID and file name
if isinstance(file_no, int):
self.file_no = file_no
else:
self.file_no = None
if isinstance(fname, str):
self.fname = fname
else:
self.fname = None
# Infer file name from memory map
if isinstance(arr, np.memmap) and (fname is None):
self.fname = arr.filename
# Data (even if None)
self.data = arr
def __del__(self):
if (self.file_no is not None) and os.path.isfile(self.fname):
try:
os.close(self.file_no)
except Exception:
pass
try:
os.remove(self.fname)
except Exception:
warnings.warn(
"Temporary file could not be removed from {}."
.format(self.fname))
def __bool__(self):
""" The implicit bool value of the Buffer object is False, when no data
is attached to it. """
return self.data is not None
def __array__(self):
""" Array interface of the Buffer class. """
return self.data
def copy(self):
if self.data is None:
data = None
elif isinstance(self.data, np.memmap):
old_file = self.data.filename
fd, new_file = tempfile.mkstemp(dir=ts.TWD, prefix="buffer_")
with open(old_file, mode="rb+") as of:
with open(new_file, mode="rb+") as nf:
shutil.copyfileobj(of, nf)
data = np.memmap(new_file, mode="r+", dtype=self.data.dtype,
offset=0, shape=self.data.shape, order="C")
else:
data = self.data.copy()
return Buffer(data, self.fname, self.file_no)
@classmethod
def _load(cls, dump):
file_no = dump.get("file_no")
fname = dump.get("fname")
data = dump.get("data")
ret = Buffer(arr=data, fname=fname, file_no=file_no)
return ret
def _dump(self):
dbuffer = {
"type": ".".join([self.__class__.__module__,
self.__class__.__name__]),
"id": str(id(self)),
"file_no": self.file_no,
"fname": self.fname,
"data": self.data,
}
return dbuffer
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# _______ _____ _____ _
# |__ __| |_ _| | __ \ | |
# | | | | | |__) | | |
# | | | | | _ / | |
# | | _| |_ | | \ \ | |____
# |_| |_____| |_| \_\ |______|
#
# Copyright (C) 2018-2020 University of Oxford
# Part of the FMRIB Software Library (FSL)
# Author: Istvan N. Huszar
# SHBASECOPYRIGHT
# DEPENDENCIES
import joblib
import inspect
import numpy as np
import collections
# TIRL IMPORTS
from tirl import exceptions as te
# DEFINITIONS
CACHE_MODES = {"memory", "db"}
# IMPLEMENTATION
class Cache(object):
""" Cache object for instance-specific use. Usage: define Cache object
as part of the initialisation, then call the three public methods (query,
retrieve, store) in succession from the same function/method.
Example:
signature = ("John", "Doe", "42", "Sheffield") # for hashing
key, exists = self.cache.query(signature)
if exists:
value = self.cache.retrieve(key)
else:
value = "successful person"
self.cache.store(key, value)
Note:
To access the same cache database from multiple functions/methods,
specify the same caller ID (the name of the function/method) while
using query(), retrieve(), and store().
"""
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ INITIALISATION ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #
def __init__(self, mode="memory", maxsize=2):
"""
Initialisation of Cache object.
:param mode: Storage mode. "memory": in RAM, "db": database.
:type mode: Union["memory", "db"]
"""
# Validate input for Cache Mode
if not mode.lower() in CACHE_MODES:
raise te.ConstructionError(
"Unrecognised GImage mode: '{0}'. Available Cache modes "
"are: {1}".format(mode, ", ".join(CACHE_MODES)))
else:
self.mode = mode