Blog Post 5: Option B

Note: I just spent the better part of an hour on Option A (Validity). It took me that long to realize that validity, credibility etc. were related to Qualitative design. I kept reading about 'measurement instrument' and wondering how I was going to make that fit my (QNT) paradigm. I think I need a few more of these classes:)

Design diagram


Random Assignment        Groups         Pretest          Intervention       Posttest          Intervention
                                           A >              O >                  X >                O >                   X
            R     >                      B >              O >                                        O

I think I am doing a quasi-experimental nonequivalent randomized-to-groups pretest-posttest design.

Non-equivalent because only one group is receiving the intervention package. Randomized-to-groups because I am randomly assigning the children to the groups and a pretest-posttest design because I am testing them before and after intervention. Not sure how to incorporate the final reversal ABAB but as long as it's included in the methods section that should suffice?

As with all 'applied' experimental designs, there are a lot of threats to internal validity:

History: Since our experiment is going to be conducted 'over-time' as well as in a classroom setting, history could play a large role. School vacations, snow days, fire alarms, teacher absences, student sickness. I had not thought about whether or not the treatment groups would be interspersed within classrooms or would be segregated. Reading about history makes me think if I separate them the chances of very different experiences in each classroom might be a major confound. On the other hand, I don't know how I could control for the 'control' group not observing the treatment package which might result in them 'learning' through observation which would skew their final scores. I'd be interested to hear your thoughts on this.

Selection: I chose a 'stratified' random selection method to insure I had equal/proportional representation across grade levels. I could control for this by de-stratifying and only using one grade level but I don't want to do that. This also brings up a similar problem as found above in history with the treatment groups being mixed in with the control groups or further 'stratified' into all or none classes. I think 'convenience' kicks in here and forces me to keep the children mixed. You can only ask so much of schools.

Maturation: I don't think this should be much, if any, of a confound as this study should take no longer than a month or two. I can see how longer studies might have to take this into account more assiduously.

Pretesting: Our pretesting procedure is an unconsequated cold probe design so there should not be an effect we're just looking at whether for not they possess the ability to be reinforced by vocal praise as a pre-condition for being in the treatment and control groups.

Instrumentation: I would say the biggest fear with instrumentation would be in either (a) the data collection piece but we would hopefully control for that with a high IOA (InterObserver Agreeement percentage) and (b), the delivery of the treatment package. Hopefully the short time frame and the inclusion of observers trained to measure fidelity would control for instrumentation errors.

Treatment replication: The treatment package is going to be 'replicated' with each student in the treatment group. This threat to validity is hopefully controlled for in the same manner that 'instrumentation' is controlled for: observation and fidelity checks.

Subject attrition: This should not be a problem due to the length of the study but is always an issue in dealing with actual students. They may move, get sick, receive ISS (in-school suspension) etc.

Statistical regression: This may be a more serious confound than I imagine. We are selecting students that are on the very 'low' end of a specific learned behavior that most children have in abundance (the ability to be reinforced through vocal praise). I'm not sure how to control for this other than mention it in the methods section? Perhaps the participants section?

Diffusion of treatment: This is a serious confound IF I keep the control and treatment populations in the same classrooms UNLESS I can deliver treatment when the control groups are somehow out of the classroom. That said, we are selecting students who do NOT demonstrate learning through vocal praise so the control group should be 'immune' from the treatment package but we are talking about not-very-well-understood social reciprocity mechanisms (despite the behavior analytic tendency to speak of topics with assuredness:)

Experimenter effects: It would be difficult to train people to deliver the treatment package and have them be 'naive' to the reason for doing so. I have seen the problems with this reality several times in my training wherein people conducting the experiment were also doing so for their doctoral projects. More than once I would be working with a student and they would be emitting low levels of correct responses. The experimenter would sit down and the student would miraculously meet criteria across two sessions and be moved to the next condition. I'm not saying it's not possible but...:) As stated above, I would try and control for this with robust IOA and fidelity checks.

Subject effects: I do not think this is confound as I highly doubt the participants would be aware of the reason they are involved in the study and that this would affect their responding.

External validity: Given that this experiment is based on very well-understood and researched principles of behavior and is being conducted in a natural setting, I would hope for a very high level of external validity and encourage folks to replicate:)

Adam

The research problem: Typically developing children in general education classrooms are reinforced by vocal praise delivered by the teacher (and other adults). Children diagnosed with autism are frequently not reinforced by vocal praise which means that prosthetic means of reinforcement are necessary (edibles, preferred items) are used to reinforce them. Many general education teachers have not been taught how to use/implement secondary reinforcement systems and the result can be moderate to severe behavior problems and general education teachers not wanting children diagnosed with autism placed in their classroom.

Hypothesis: Using classic Pavlovian conditioning procedures, students diagnosed with autism will be conditioned to be reinforced by vocal praise through an observational learning procedure. That is, they will learn to be reinforced by vocal praise through observing other students receive reinforcement paired with praise.

Population/participants: Children diagnosed with autism (Ages 2-10)

Selection of Participants/Data collection: This is an experimental reversal design (ABAB). I'm going to have a stratified sampling group that is divided by age group. Participants will be randomly selected from each grade level. I'll randomly assign participants to treatment and control groups. Treatment groups will be reinforced for observing vocal praise serve as reinforcement for their peers while the control group will not.

Extra Credit 1

Hall, R. V., Lund, D., & Jackson, D. (1968) Effects of teacher attention on study behavior. Journal of   Applied Behavior Analysis, 1, 1-12

Study Design: This was a quasi-experimental multiple baseline across participants design. The authors used purposive non-probability sampling. The participants were recommended by their respective principals due to their 'disruptive' or 'dawdling' behavior. It was an ABAB design with a post-check to determine generalization.

Participants: The studies were conducted in two elementary schools in low SES areas of Kansas City, MS. Teachers were selected by their principals and the students were nominated by their teachers. It's interesting to see how much information was offered in 1968. Here is a direct quote, "Robbie was chosen because he was considered a particularly disruptive pupil who studied very little". "Rose" was a classmate of Robbies and was selected because of her poor study behavior which included lying her head on her desk, taking off her shoes, talking and getting out of her seat. Ken and Gary were also chosen due to their disruptive behavior."Joan" was chosen due to her 'dawdling' behavior and finally "Levi" also emitted high levels of disruptive behavior. 

Data Collection: Data was recorded on, "recording sheets lined with triple rows of squares. Each square represented an interval of 10 sec.". Row 1 was used to record student behavior, row 2 was teacher verbalizations and row 3 was teacher proximity. A symbol was used for notational purposes.

Data Analysis: Data was graphically displayed by transferring data from the data collection sheets to graphs. Percent of study behavior, intervals of teacher attention and percent of disruptive behavior were graphically displayed. Interobserver agreement (IOA) was analyzed by having a second observer periodically made simultaneous observation records. Agreement was checked interval by interval. The percentage of the of agreement of the records [# of agreements X 100 /(# of agreements + # disagreements)] yielded the percentage of IOA

Conclusion: The study found that contingent use of teacher attention, "..can be a quick and effective means of developing desirable classroom behavior". It was found that many of the "poor study" students were being reinforced for poor study habits and that when the contingencies shifted and they received teacher attention for studying their study habits improved. It was noted that these teachers did not have formal training in reinforcement principles and generally had good instructional control over their classrooms. The teachers found the reversal phases aversive and were happy when they went back to reinforcing good study habits. They finally noted that the materials the students were being asked to study were within their skill set and that if this were not the case then gains would be unlikely. Due to the fact that they incorporated not one but two reversals I do think the conclusions are valid.

Rehfeldt, R. A., & Root, S.L. (2005) Establishing derived requesting skills in adults with severe developmental disabilities. Journal of Applied Behavior Analysis, 38, (101-105)

Study design: This was also a quasi-experimental design but this was a multiple probe design across participants with repeated probes for derived stimulus relations and derived requesting skills.

Participants: There were three participants who were diagnosed with severe mental retardation and were reported to have "rudimentary or no functional communication skills". "Sam" was 20 years old with a reported IQ of 21. Kenny was 34 years old with a measured IQ of 30 and Carl was 27 years old with an IQ of 25. He was also nonvocal. All three took medication. The medication doses were stable throughout the study.

Data Collection: The dependent measures were the proportion of correct responses during derived request and derived relations probes. Average IOA was 98% across all phases. The materials used included: a three-ring binder with 4 strips of Velcro. digital photos of preferred items, text of the preferred items and a stimulus placement board.

Data Analysis: Data was analyzed visually by graphing all of the results.

Conclusions: The authors stated that the results,"...illustrate that a reinforced history of relational responding is sufficient for the emergence of derived requesting skills in individuals with severe developmental disabilities". They went a little further by stating, "a history of reinforced relational responding may facilitate the emergence of novel forms of requesting and other verbal skills." This is a bold claim as the production of 'generative language' skills is the holy grail of any special educator. I am not going to engage in the debate raging about whether or not 'Relational Frame Theory' is useful or just is a re-packaging of stimulus equivalence theory but I will say that this study is not designed to support the claim that the participants emitted 'novel' forms of requesting. They did demonstrate that a well-known intervention is effective on adults with severe disabilities though.


(I put these here not to pad my word count but to follow the letter of the prompt:)


Hall, R. V., Lund, D., & Jackson, D. (1968) Effects of teacher attention on study behavior. Journal of   Applied Behavior Analysis, 1, 1-12

 

Rehfeldt, R. A., & Root, S.L. (2005) Establishing derived requesting skills in adults with severe developmental disabilities. Journal of Applied Behavior Analysis, 38, (101-105)