The problem of content validity

CHAPTER II. THE PROBLEM OF CONTENT VALIDITY
2.1. The practical values of content validity in validation process.
When it comes to developing measurement tools such as intelligence tests, surveys, and self-report assessments, validity is important. A variety of types of validity exist, each designed to ensure that specific aspects of measurement tools are accurately measuring what they are intended to measure and that the results can be applied to real-world settings. Before we move into discussing content validity, it is important to understand that validity is a broad concept that encompasses many aspects of assessment. For example, face validity describes the degree to which an assessment measures what it appears to measure, concurrent validity measures how well the results of one assessment correlate with other assessments designed to measure the same thing, and predictive validity measures how well the assessment results can predict a relationship between the construct of being measured and future behavior.
So, what about content validity? Content validity refers to how accurately an assessment or measurement tool taps into the various aspects of the specific construct in question. In other words, do the questions really assess the construct in question, or are the responses by the person answering the questions influenced by other factors?
In the previous versions of the Standards, validity evidence based on test content was described as “content validity,” and this term was also common in the psychometric literature. Lennon provided an early defi nition of content validity as the extent to which a subject’s responses to the items of a test may be considered to be a representative sample of his responses to a real or hypothetical universe of situations which together constitute the area of concern to the person interpreting the test. Sireci provided a broader definition that included aspects of test development. He described content validity as pertaining to four elements of test quality: domain definition, domain representation, domain relevance, and appropriateness of the test development process. Although there is long consensus that these four elements are important for evaluating the use of a test for a particular purpose, many validity theorists claimed “content validity” was not a technically correct term because validity refers to interpretations of test scores and not to the content of an assessment. We see the theoretical logic in that argument; however, we, like Ebel and Yallow and Popham believe the term “content validity” is useful for practitioners and lay audiences and effectively communicates an important aspect of the quality of test scores. We define content validity as the degree to which the content of a test is congruent with testing purposes. In addition, we use the terms “validity evidence based on test content” and “content validity evidence” interchangeably. Essentially, the “debate” over the term content validity is one of nomenclature and is likely to persevere in academic circles. However, what will also persevere is the importance of affi rming that the content of a test represents its intended construct and is appropriate for accomplishing the testing purposes.
Content Validity Measurement
So how is content validity measured? How do researchers know if an assessment has content validity? Content validity is most often measured by relying on the knowledge of people who are familiar with the construct being measured. These subject-matter experts are usually provided with access to the measurement tool and are asked to provide feedback on how well each question measures the construct in question. Their feedback is then analyzed, and informed decisions can be made about the effectiveness of each question.
Examples:
To better illustrate the significance of content validity, let's look at two examples. One example explains how content validity can be helpful in a clinical setting, and the other in a business setting. Assessment and measurement tools like surveys and questionnaires are quite common in the social and behavioral sciences. Content validity is a critical aspect of developing tools that can help practitioners understand and treat behavioral and mental health conditions.


For example, if a particular assessment tool is designed to measure the severity of symptoms of clinical depression, a group of psychiatrists would evaluate each question and provide an opinion or rating on how well the wording of each question taps into measuring the severity of depression symptoms. The independent ratings of each subject-matter expert are then compared and analyzed to determine the degree of content validity that exists for each question. The assessment developers can then use that information to make alterations to the questions in order to develop an assessment tool which yields the highest degree of content validity possible.