Sunday, April 24, 2011

Critique of the Motivation Toward the Environment Scale - by Lauren Campbell

(White paper on cultural relevance in MTES)

tripwiremagazine.com
The growing awareness of our environmental issues is reaching an all-time high. As our global population reaches critical mass, our environmental awareness is crucial to our existence. Although the concerns for our environment are on the rise, the majority of the us seem to still lack the motivation needed to behave pro-environmentally. This lack of environmental motivation, at a time of near crisis threshold, is detrimental to the livelihood of our planet and the continuation of our species. With many pressing environmental issue looming, a small handful of researchers in Canada created the Motivation Toward the Environment Scale (MTES), a psychological test that would help examine why some people are motivated to behave pro-environmentally and why others are not.

The MTES was developed by Pelletier, Green-Demers, Tuson, Noels, and Beaton (1998) and was published in the Journal of Applied Social Psychology. There were no specific information regarding the testing universe and the development of this test was mostly influenced by the previous research on environmental attitudes and the rising social phenomenon of environmental concern (Pelletier, Green-Demers, Tuson, Noels, & Beaton, 1998). Séguin, Pelletier, and Hunsley (1999) have shown in previous research that environmental concern in addition to environmental education did not always lead to pro-environmental behavior. Pelletier et al. (1998) constructed this scale based upon Self-Determination Theory (SDT) that was proposed by Deci & Ryan in 1985 (as cited in Pelletier et al., 1998); they used the basis of SDT, as well as the motivation continuum proposed by SDT, in development and in the defining the constructs for the MTES. To date, this test was not empirically-based nor does it contain norms. Pelletier et al. (1998), did not appear to ask experts to rate their opinion of importance as it appears that this topic of study did not have a large amount of prior research at the time.

The MTES was modeled after the SDT and the different types of motivation developed in that theory. The different forms of motivation are located on a continuum, but are based from three different types of motivation: Intrinsic motivation, Extrinsic motivation, and Amotivation. Extrinsic motivation is further scaled into four categories: external regulation, introjected regulation, identified regulation, and integrated regulation. These types of motivation are located on a continuum where intrinsic motivation is the most self-determined and amotivation is the least self-determined. There were some pilot testing completed for this scale and there were three studies conducted prior to publishing the scale to establish internal consistency, test-retest reliability, and construct validity (Pelletier et al., 1998).Through this pilot testing researchers were able to narrow down the questions from 60 items to 24 items.

According to Pelletier et al. (1998) the scale had been set at 6 subscales with 10 questions each, but after the first study, which was used to determine whether the proper constructs of motivation were being measured, the scale was narrowed down to 4 items within each of the 6 subscales. The test is formatted in a Likert Scale style. The scale ranges from 1 (does not correspond at all) to 7 (corresponds exactly) with 4 (corresponds moderately) being in the middle. The test is scored by adding up all the scores within the subscales and acquiring the mean score for each individual subscale. The subscale with the highest average is then considered to determine what type of motivation the test taker has toward the environment. The MTES contains objective items, but there are very little specific instructions for the test taker or administrator. Other studies showed that scoring the MTES can be different for different studies and MTES does not have standardized administration to my knowledge. Villacorta, Koestner, and Lekes (2003) used the average score of each subscale for the purpose of their research, whereas Green-Demers, Pelletier, and Ménard (1997) used very elaborate scoring of the MTES to compute global self-determination indices and the questions were assigned different weights.

The MTES is mostly used in research settings, but it could also be used in educational settings. Most often the MTES is given in an academic setting to college students, but it has also been mailed to people’s homes. While the student take the test on campus, those who receive the test in the mail complete the test at their place of residency and return it by mail (Pelletier et al., 1998; Séguin , Pelletier, & Hunsley, 1999). There is at least one study where participants were asked to come to a lab to take the MTES (Villacorta, Koestner, & Lekes, 2003). The test administration does not seem to have contingencies regarding administration in a group or individual setting, but one might assume that by administering in a group format, where others have access to the responses of others, social biases could arise. The MTES appears most often as a pencil and paper test, but whether or not an electronic test is available cannot be said for sure; information regarding test protection could also not be obtained.

The need to take this test seems to be purely research based and there is not a time where someone needs to take the test. I would argue that with the growing concern regarding our environment, the MTES and other such environmental tests, that help us understand our environmental knowledge, motivation, and pro-environmental behavior, are essential to our survival. There is serious need for these types of tests in the educational system to assess a person’s environmental intelligence to assess that person’s role in the type of environmental classroom or social situation.

The MTES does not have a set age range. It requires the test taker to be able to read and to have enough environmental knowledge to understand words such as recycling, reducing, reusing, etc. This is an age restriction in the sense that young children cannot typically read and may not have the conceptual knowledge needed to fully understand environmental and motivational terms. Most testing with this scale has been done at the college level with the mean age between 21 to 23(Green-Demers, Pelletier, & Ménard, 1997; Pelletier et al., 1998; Villacorta et al., 2003). When the test was sent in survey form through the mail, the mean age was rougly 49 years-old (Séguin et al., 1999). There does not seem to be any certain demographic for which this scale would not be suitable. There might be discrepancies as to how different ethnicities and cultures view the environment, so it could be argued that the interpretation of this scale may not be same across all demographic divides; not that it would be inappropriate, but just misinterpreted. It would be useful to conduct additional research in the area of culture and environmentalism, to see if different perceptions occur in the first place especially in collectivistic vs. individualistic societies. There have not been norms set for the MTES because of a lack of research in this domain and it would be interesting to see whether this changes in the future.

Evidence for reliability is high for the MTES. There has been more than one study to verify its test-retest reliability and its internal consistency. According to Pelletier et al. (1998), The MTES shows very acceptable levels of reliability and when broken down into the six subscales of motivation that the MTES measures, the internal consistency of those subscales was considered satisfactory with values ranging between .78 and .96 (.78 < Cronbach’s α < .96). The MTES has showed satisfactory test-retest reliability over a 5 week period; Cronbach’s α correlation values ranged from .63 to .79, p < .01 (.63 < Cronbach’s α < .79) (Green-Demers et al., 1997; Pelletier et al., 1998; Séguin et al., 1999). The testing of the reliability of the MTES was appropriate and has occurred in more than one study. Both internal consistency and test-retest reliability have been tested. Reliability regarding alternate forms has not been done due to a lack of alternate forms of the MTES.

There appears to be evidence for construct validity, convergent validity, and discriminant validity. Evidence of criterion-related validity or content validity is scarce and could not be found for this scale. Convergent validity for the MTES and its sub-scales was supported. According to Villacorta et al. (2003), convergent validity was established by comparing the MTES to other self-reports that measure regulatory styles in the environment and in the academic domains (r(141)= .35, p <.01 and r(142) = .32, p < .01 respectively). There was also some convergent validity shown in Pelletier et al. (1998) with moderate correlations between the MTES and environmental behaviors. (.14 < Cronbach’s α < .48, p< .05 and p < .01). These environmental behaviors were positively correlated with the motivational continuum that SDT formulated. Discriminant validity was also shown through these procedures. In the process of showing convergent validity, Pelletier et al. (1998) showed that there was discriminant validity in the correlations of the MTES and the amotivation and external regulation categories of the Self-Determination Theory (.-12 < Cronbach’s α < .-25, p <, .05 an p < .01). These types of motivations are non-self-determined, so it was hypothesized that there would be discriminant validity in regards to these areas. The coefficients were also used to determine construct validity; the environmental behaviors acted not only as a comparison for the MTES for convergent and discriminant validity, but were also a validation for the construct that the MTES is measuring. Pelletier et al. (1998) also used confirmatory factor analysis, which the MTES endured, to confirm the construct validity. It has also been shown through exploratory and confirmatory factor analyses that the factorial structure of the MTES is supported; the relationship between the MTES sub-scales and psychological constructs provides construct validity (Green-Demers et al., 1997).

The MTES does seem to have face validity, although the test taker may not be aware that it is measuring different aspects of motivation. The questions are written such that a test taker could recognize that the MTES is asking about environmental behaviors. There is no direct information regarding content validity or any statistical references, but the MTES was put through stringent analyses to provide indirect evidence of content validity. The pilot tests that were ran by Pelletier et al. (1998) were conducted such that the questions in the MTES were a reflection of the Self-Determination Theory; which provides some evidence of content validity. The same goes for criterion-related validity. There was not any statistical references regarding this type of validity, but again, since the MTES was developed around Self-Determination Theory, one could argue that the criterion-related validity, more specifically concurrent validity, could be assumed. For example, for the people who score high on the MTES in intrinsic motivation, researchers would assume that it is those test takers who are out there behaving pro-environmentally and those who score high on amotivation would be the test takers who are not behaving, or behaving the least, pro-environmentally. The evidence for the criterion-related validity and content validity are not very accurate. These are just validity assumptions that someone could make based on their knowledge of all types of validity; there were not any statistical information regarding these two types. Construct, convergent, and discriminant validity did have statistical references and were focal points of conducted research. This allows test administrators and test takers to know the levels of validity for the MTES. These types of validity are accurate. There has not been any evidence of reliability and validity limiting usage of the MTES to different populations. It could be argued, however, that the MTES is limited to the use of motivational research or behavioral research because it is built around the Self-Determination Theory. This limitation could definitely hinder the use of the MTES in different areas of research.

The decisions that are made with this test are decisions regarding a test taker’s motivation to behave pro-environmentally. The MTES also narrows down what type of motivation the test taker is using to behave pro-environmentally based upon the Self-Determination Theory. Test scores could be misinterpreted if someone did not fill out the scale properly or if the test taker comes from a different culture. Different cultures may perceive pro-environmental behavior differently than how the MTES has been designed to assess those behaviors. In regards to Ethical Standard 9.06-Interpreting Assessment Results, this concept should be something that most test scorers and interpreters keep in mind. According to Pelletier et al. (1998), external validity needs to be more established with the MTES; when interpreting assessment results this should be kept in mind. This is also directly related to 9.09 (b) in the ethical standards of assessment-test scoring and interpretation services. Because the MTES is built upon the Self-Determination Theory, knowledge of this theory should be gathered prior to administering the scale. Based on ethical standard 9.05-test construction, it is the responsibility of the MTES developers to recommend prior understanding of SDT; which, are available in all the studies cited for this paper (Green-Demers et al., 1997; Pelletier et al., 1998; Seguin et al,. 1999; Villacorta et al., 2003). The information given by these researchers also adheres to the ethical standard 9.01 (b). By not having knowledge of this theory, the interpretation of the results could be skewed and misinterpretation of the results could follow. Misinterpretation of the results could lead to consequences such as misidentifying the motivation of the test takers; this could lead to the assumption that the test taker is not motivated to behave pro-environmentally when in fact they are. This could also lead to the wrong data being published, which would be detrimental to the area of research and the body of human knowledge in general. An unethical administration of this test could also leave a person emotionally taxed. If a test taker was someone who identified with behaving pro-environmentally and then received test results what stated otherwise due to an unethical administration, then that person could feel bad about his or her self. This would also be breaking Principle A and C of the APA Ethical Principles.

MTES is also susceptible to social desirability; however, according to Pelletier et al. (1998), the MTES seems to be unaffected by social desirability. If a test taker did give fake responses then it could skew research results, but there is not any direct reason that a test taker would want to do this. If a researcher were to explain what the assessment is measuring and why, which according to ethical standard 9.10 should be done, then a test taker might even be more interested in answering correctly, so that he or she knows where they stand in regards to motivation and the environment. Random responding might also be a challenge, although theoretically it is a challenge in all test taking domains. This could occur with a test taker who is not interested in taking the MTES or who does not understand the vocabulary of the MTES. Guarding against random responding could be prevented if the test administrator followed ethical standard 9.03 (a) and (b) and 9.07. Social desirability could also be prevented if a test administrator followed ethical standard 9.03 (a), 9.04 (a), and 9.11 and could guarantee confidentiality and test security. In order to follow the ethical standards that the APA expects, a professional is needed to administer and interpret results. By gaining adequate knowledge of the construct being measured and of the test taker, the test administrator should be able to properly administer and interpret the results. These are standards that the APA has put in place for a reason. Someone who did not have previous knowledge of SDT, and did not have the proper means to interpret the MTES correctly, could cause emotional harm to a participant/test taker.

Overall, I think that the MTES is a step in the right direction. The need for more environmental research in the psychological domain is something that cannot be ignored. With the growing evidence of society’s lack of motivation toward pro-environmental behavior and the issue of sustainability looming over our heads, we must learn why people are not taking the proper action. The MTES offers a glimpse into the psychological constructs that need investigated in order to accomplish a change in environmental mentality. It has satisfactory reliability and validity, which have been shown through multiple studies. There was a discrepancy in research with publishing dates; it appears the test development research Pelletier et al. (1998) was published after one other study, Green-Demers et al. (1997), in which the MTES was used as a measurement tool. This is probably due to the rate of publishing, but should be noted. There does not seem to be a large amount of ethical concern for the MTES, but research should be done in regards to different cultures and different perceptions of pro-environmental behavior. Further research in the field of psychology concerning environmental attitudes must be done. The Motivation Toward the Environmental Scale is one of few environmental measures offered in research and needs to be developed upon in order to progress this topic. I will definitely see myself using this scale in the future in order to build on psychological science in the environmental and motivational domains.

References

Green-Demers, I., Pelletier, L. G., & Ménard, S. (1997). The impact of behavioural difficulty on the saliency of the association between self-determined motivation and environmental behaviours. Canadian Journal of Behavioural Science, 29, 157-166. doi: 10.1037/0008-400X.29.3.157

Pelletier, L. G., Green-Demers, I., Tuson, K. M., Noels, K., & Beaton, A. M. (1998). Why are you doing things for the environment? The motivation towards the environment scale (MTES). Journal of Applied Social Psychology, 28, 437-468. doi: 10.1111/j.1559-1816.1998.tb01714.x

Séguin, C., Pelletier, L. G., & Hunsley, J. (1999). Predicting environmental behaviors: the influence of self-determined motivation and information about perceived environmental health risks. Journal of Applied Social Psychology, 29(8), 1582-1604. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/j.1559-1816.1999.tb02043.x/abstract

Villacorta, M., Koestner, R., & Lekes, N. (2003). Further validation of the motivation toward the environment scale. Environment and Behavior, 35, 486-505. doi: 101177/0013916502250753

1 comment:

  1. Great article, worthy of publish, well sourced and balanced.

    Do you have access to a complete Motivation Toward the Environment Scale?

    ReplyDelete