Commit 91f4ac73 authored by Paul McCarthy's avatar Paul McCarthy 🚵
Browse files

DOC: changelog

parent 872af70a
......@@ -25,13 +25,27 @@ Changed
* FUNPACK will now parallelise tasks by default; previously it would only
parallelise tasks if ``--low_memory`` mode were selected. Data import,
cleaning functions, processing functions, the ``removeIfRedundant`` process,
and TSV exporting all parallelise their work. New columns created by processing
functions are saved to disk, and re-loaded by the main process, rather than
being passed back to the main process via inter-process communication.
parallelise tasks if ``--low_memory`` mode were selected.
* The data import stage is parallelised by using multiple processes to read
different chunks of the input file(s), and then concatenating the resulting
``pandas.DataFrame`` objects afterwards.
* Cleaning functions are executed on each variable in parallel.
* Each processing step is executed in parallel where possible
(e.g. ``independent`` processes), but processing steps are still executed
sequentially. New columns created by processing functions are saved to
disk, and re-loaded by the main process, rather than being passed back to
the main process via inter-process communication.
* The ``removeIfRedundant`` process now compares pairs of columns in parallel.
* The data export stage is parallelised by writing chunks of rows to different
files, and then concatenating them into a single output file afterwards.
* The ``--variable``, ``--subject`` and ``--exclude`` options now accept
comma-separated mixtures of IDs and MATLAB-style ranges.
* Updates to FMRIB categories.
* Updates to FMRIB processing rules, to take advantage of parallelism.
* The ,:mod:`icd10` module must now be initialised via the
:func:`.icd10.initialise` function, when it is to be used in a multiprocessing
context. This is not necessary when ``funpaxk`` is configured to not
parallelise tasks (e.g. with ``--num_jobs 1``).
Deprecated
......@@ -39,7 +53,8 @@ Deprecated
* The ``--low_memory`` and ``--work_dir`` options have been deprecated, and no
longer have any effect.
longer have any effect. The :mod:`.storage` module is no longer used, but is
still present for possible future usage.
1.4.5 (Thursday 5th December 2019)
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment