ABSTRACT
To illustrate this situation, consider the following example. Back in the 1990s a school of pharmacy was working on developing a new method of delivering its recently developed Pharm.D. curriculum to B.S. pharmacists desiring to obtain this degree, but unable to take a one-or two-year sabbatical to return to school. Therefore, the school worked with different delivery systems to provide distance learning for the didactic portion of the course work. The primary investigator developed a satisfaction index on a linear scale for the pharmacist to evaluate the convenience, flexibility, and usefulness of the course materials, as well as the user friendliness of the course work. It was assumed that the better the response (maximum score of 10), the more likely that pharmacists would begin the course work and continue to the end of the didactic portion of the program. A pilot study was conducted on a random sample of pharmacists. Two different delivery methods were considered: written monographs (M1) and computer-based training using CD-ROMs (M2). However, early in the development of course work there were concerns that institutional (primarily hospital) and ambulatory (mostly retail) pharmacists might possess different learning styles and might react differently with respect to their evaluation of the course materials. Therefore, the pilot study was designed to evaluate two independent variables, the delivery system used, and the pharmacist’s practice setting, either institutional (S1) or ambulatory (S2). This can be illustrated in the simplest possible experimental design, a two-by-two (2 × 2) factorial design:
Methods M1 A B
M2 C D
S1 S2
Settings where A, B, C, and D represent the mean results for the continuous dependent variable (satisfaction index), and rows and columns represent the main factors tested. For example, A represents the responses for pharmacists practicing in institutional settings who receive the written monongraphs; whereas D represents ambulatory pharmacists exposed to computer-based training. In this design the principal investigator (PI) was interested in evaluating the main effects of both the factors used and the interaction of these two factors. In this case the PI was dealing with three different hypotheses: H01: μM1 = μM2 (Main effect of the delivery method) H02: μS1 = μS2 (Main effect of the practice setting) H03: (μM1,S1 − μM1,S2 ) = (μM2,S1 − μM2,S2 ) (Interaction between method and setting) The first hypotheses (H01) evaluates the main factor for the two methods used for distance learning (M1, M2). Are they approximately the same or are they statistically different? The second hypothesis (H02) assesses the influence of the pharmacists’ practice setting (S1, S2) and what influence settings might have on evaluations of the course materials. These first two hypotheses are called tests of main effects and are
similar to separate tests using a one-way analysis of variance. The third hypothesis (H03) evaluates the possibility of relationships between the row and column variables. As discussed below, two independent variables are considered to interact if differences in an outcome for specific levels of one factor are different at two or more levels of the second factor. Whenever we evaluate the effect of two or more independent variables on a dependent variable, we must be cautious of a possible interaction between these independent variables. The interaction effect measures the joint effects of two or more factors on the dependent variable. If the factors are independent of each other, or have no relationship, there will be no interaction. We are interested in detecting interactions because the overall tests of main effects, without considering interactions, may cause us to make statements about our data that are incorrect or misleading. The validity of most multifactorial designs is contingent on an assumption of no interaction effects among the independent variables. One might argue that a more appropriate procedure is to test for any interaction first and if no interaction is detected (e.g., the test is not significant), then perform separate tests for the main effects. However, if interaction exists, it is meaningless to test the main effects or to try to interpret the main effects. The approach used in this book is to evaluate the main effect and interactions in concert as a more efficient and time-saving method. Granted, if the interaction is found to be significant, the results of the evaluation of main effects are without value, because the factors are not independent of each other. To illustrate the various outcomes, consider the possible outcomes in Figure 12.1 for our experiment with the factors of delivery system and practice setting. Here results are plotted for one main factor (setting) on the x-axis and the second main factor (delivery system) on the y-axis. In Outcome I the results are the same for all four observations; therefore the investigator would fail to reject any of the three hypotheses and conclude that there was no significant effect for either of the main effects and there was no interaction between the two factors. For Outcome II, there is a significant difference between the two delivery methods used (M1 > M2) and the investigator could reject H01, but would fail to reject the other two hypotheses. The opposite results are seen in Outcome III, where the investigator would find there is a significant difference between the two practice settings (S2 > S1) and reject H02, but would fail to reject the other two hypotheses. Outcome IV represents a rejection of both H01 and H02 where M1 > M2 and S2 > S1, but there is no significant interaction and H03 cannot be rejected. Outcomes V and VI illustrate two possible interactions. In Outcome V there are significant differences in the main effects and a significant interaction between the two main factors. We can see that the two lines cross and there is a significant interaction between methods and settings. In this example, it appears that institutional pharmacists prefer monographs and ambulatory pharmacists favor the computerbased training. Because of the interaction, it becomes meaningless to evaluate the results of the main effects, because if there was a significant difference between methods of delivery, it may be influenced by the practice setting of the pharmacists. In Outcome VI there is no difference in M1 based on the practice setting, but there is a difference for M2. Is this significant? Is there an interaction between the two main effects (factors)? A two-way analysis of variance can determine, with a certain degree of confidence, which hypotheses should be rejected as false.
