Custom Search

Analysis of the Lroc2 System

Analysis of the Lroc2 trading system illustrates the RM Anova method. The system includes two parameters, "fast” and "slow” lengths.

This system was used to trade n = 60 high technology stocks from March 1998 to March 2000, and the   returns computed as the Total Profit without allowance for a commission. For each stock, tests were repeated   for each of m = 6 parameter sub-sets: Fast = [5, 10] bars; Slow = [20, 30, 40] bars.

Prior trading experience with the Linear Regression Slope (Lroc) indicator suggests that the [10, 40] sub-set   may have particular value. Prior to computing the returns we plan to compare the [10, 40] sub-set   performance against all other sub-sets with a set of "repeated” contrasts.

The accompanying chart summarizes the test results. The average return over all stocks, upper 95%   confidence limit, and lower 95% confidence limit of the returns for each parameter sub-set are plotted. The   highest total profit is achieved by set no 6, with a Fast length of 10 bars and a slow length of 40 bars. Is this   particular parameter combination truly optimal, and is the system valid?

The data was analyzed with a univariate repeated measures Anova procedure. The result was: F (df = 5; 59)   = 2.16 significant at a level of 0.065 (including a small correction for lack of sphercity). Most traders would   consider this result as highly reliable. The values of all means were individually significant at levels   over 0.0001. In summary the system was validated by the RM Anova procedure. Was sub-set 6 = [10, 40] truly   optimal? The contrast for the sub-set 6 = [10, 40] mean vs. sub- set 2 is significant at a level of 0.017 and the   contrast vs. sub-set 3 is significant at 0.07. All other pair-wise comparisons have significance levels > 0.1.   Therefore sub-sets 1, 4, 5 & 6 appear to be "Optimal”. Note that if we did not plan to test sub-set 6 against the other sub-sets prior to collecting the data, this conclusion would be unfounded.

Without a pre-experiment plan, we must correct the contrast significance levels for data-snooping with a   multi-comparison adjustment. A Bonferroni adjustment to the contrast significance levels for the sub-set 6   vs. sub-sets 2, and 3 pairs would balloon the significance levels to 0.226 and 0.66 respectively. With data- snooping the system has no demonstrated optimum and cannot be valid. We would be forced to revise our   prior validity conclusion based upon the RM Anova results.

The Lroc2 system analysis dramatically illustrates the importance of a pre-test formulation of contrasts,   based upon prior experience with the trading system. The expectation of continued superior performance of   the highest tested sub-system can depend upon the trader`s prior knowledge of the system performance. As   in all analysis, prior knowledge greatly amplifies the power of statistical tests.