Why is it concerning that the analysis relies entirely on Matlab toolboxes?
1. Reproducing the analysis requires access to the exact toolbox with the exact version number. May also have platform dependencies (e.g., outputs slightly different results on macos, linux, and windows).
2. Often the version of the toolbox is not reported. It isn't in this paper.
3. Increased use of the toolbox may lead to price gouging, excluding many who can't afford to purchase it. Besides the unfairness, this impacts the ability of just about anyone to reproduce the analysis from the raw data.
4. After some years the toolbox may not be available at all, for example if the company considers its maintenance a burden with negative returns on investment.
If you are a reviewer or an editor: at the very least demand the reporting of the toolbox version and the operating system version where it was run. And ideally demand enough specification of the algorithms used by the functions provided in the toolbox that an independent reimplementation and reanalysis is possible.
#reproducibility #academia