Skip to content

Plausable fix for underflow and masking problems

Tom Nichols requested to merge nichols/randomise:FDRprecisionFix into master

FDR's --othresh option has two bad side effects, both resulting from internally converting to 1-minus p-values.

First, the 1-minus creates underflow problems, such that small but non-zero p-values become exactly zero when passed through the 1-(1-p) transformation.

Second, the 1-minus transformation leaves the background with a value of 1.0, which, while not a problem strictly, is just bad practice.

The fix I've proposed simply increases the precision, but ideally the logical would be revised so that the 1-minus p tranformation would be unncessary. However, I haven't even tried to compile this... no idea if this is the best even quick fix.

@mwebster

Merge request reports