Experimental political science is the use of experiments, which may be natural or controlled, to implement the scientific method in political science.
History development and usage
In the 1909 American Political Science Association presidential address, A. Lawrence Lowell claimed: “We are limited by the impossibility of experiment. Politics is an observational, not an experimental science….” He argued that political science, as an emerging discipline, did not need to follow the experimental-led approach of the natural sciences. In the 1900s, observational research was the only way of doing research in political science.
The first experiment in political science is regarded to be Harold Gosnell's 1924 study on voter turnout in Chicago. In this experiment, he randomly assigned districts to receive information on voter registration and encouragement to vote.
In the 1950s, the behaviorist revolution was in full swing, and the development of experimental politics ushered in the first watershed.
The first experiment published in the American Political Science Review was in 1956, 50 years after the publication's inception. The American Political Science Review covered all fields of political science and was found by the Cambridge University Press in 1906. Another mainstream political science journal, the Journal of Conflict Resolution, also began publishing articles on experimental research during this period.
In the 1970s, with the rise of political psychology, the rejection of experimental politics softened.
In the 1980s, computer-assisted telephone interviewing began to appear and was used for the collection of experimental data—the advances in the technology led to the initial rise of experiments.
The period before 2000 was classified as the prelude to the experimental era. This is the long incubation stage of experimental politics. During this period, experimentation was not the main academic activity of political science. Many great discoveries come from the integration of scholars with using multiple methods.
From 2001 to 2009, this period was the first generation when experiments were widely used. Experimentation was becoming part of the political scientist's toolkit during this period due to the development of information technology.
From 2010 till now, we are in the new era of experimental political science 2.0.
Due to the development of the internet, the emergence of commercial internet survey panels and crowdsourcing platforms make data much cheaper and easier to obtain than ever before. Abundant and accessible information is the basis for the wider implementation of experiments in political science.
In 2010, the Experimental Research section of the American Political Science Association held its first conference.
In 2012, Bond et al.'s experiment delivered political mobilization messages to 61 million Facebook users. They aimed to explore whether an “I Voted” widget that announced one’s election participation to others increased turnout among Facebook users and their friends. Today's mature social media provide the opportunity to intervene experimentally with a vast capacity population.
In 2014, the first issue of the Journal of Experimental Political Science was published by the Experimental Research section of the American Political Science Association.
From 2010 to 2019, there were 75 articles using experimental methods published in American Political Science Review. The figure was 76 from 1950 to 2009 in this journal.
Current experts in experimental methodology in political science include Rebecca Morton and Donald Green.
Among the areas that it is used in are:
- Political psychology, including survey methodology, loss aversion, influences on voter turnout, and media influence;
- The effects on outcomes of different voting systems (e.g., storable votes), including the study of tactical voting;
- The political economy of development;
- International relations
- Interactions and voting in legislatures (and other deliberative assemblies), including the effects of different voting methods and control of the agenda.
What makes a good experiment?
Validity
Social scientists, including political scientists, have long used validity to measure whether a particular analytical method can provide credible evidence for validating theoretical inferences. Donald Campbel defines validity as the validity of an empirical research design or method as the degree of approximation between the knowledge inference based on the design or method and the actual situation. Alternatively, validity is the extent to which we can believe that the empirical inference can reflect the real laws of human society. This definition has been recognized by most scholars.
Validity can be further divided into internal validity and external validity. Campbell further refined the internal validity into three parts: constructive validity, causal validity, and statistical validity.
Classification of validity aims to facilitate researchers to describe and conduct research from different aspects. Validity itself is a holistic concept. Any single class cannot exist in isolation from other validity. For example, high construct validity means that the research design has a higher degree of fit with the theory being studied. This makes the relationship of causal variables more stable from the statistical perspective, which supports the improvement of statistical validity.
Internal validity is a prerequisite for external validity. If a reasonable estimate of the target group is not made, there is no point in extending relevant estimates to groups outside the target group.
Internal validity
Internal validity refers to the degree to which knowledge inferences based on empirical research approximate the actual attitudes or behavioural patterns of the target population.
Constructive validity
Construct validity involves the generality of empirical inferences and aims to evaluate whether a research design is a reasonable and targeted assessment of the target theory.
Causal validity
Causal validity is similar to the "identification problem" in economics, which examines whether an empirical design can effectively eliminate interference factors and provide accurate evidence for determining causal effects or mechanisms.
Statistical validity
Statistical validity refers to whether there is a significant and stable statistical relationship between the core causal factors of the study at the empirical level. The most common test of statistical validity is repeated testing of the same sample of the target population.
External validity
External validity refers to how empirical inferences apply to populations other than the target population.
Experimental designs
Laboratory experiment
Laboratory experiments place subjects in specific environments and examine how individuals make specific political decisions (such as voting, jury trials, and legislation). Laboratory experiments have stricter control over the experimental site and time, and the entire experimental process must be completed under the supervision and guidance of the researcher. Questionnaires are usually used to collect the subjects' personal information and experimental results.
Laboratory experiments are usually carried out in independent laboratories, reflecting the researcher's understanding and attitude towards the information contained in time and space. Laboratory experiments emphasize the control environment and other non-experimental elements to maximize the exclusion of interference factors and accurately measure the causal effects of researched factors. The laboratory itself does not refer to the specialized laboratory used in the natural sciences. Classrooms, activity rooms, or other independent spaces can be used as experimental places.
Conjoint survey
The conjoint survey experiment is a method for examining multidimensional preferences.
Lab-in-the-field experiment
At one extreme, "pure" laboratory experiments are carried out in environments that researchers highly control. Students are often a convenient sample in lab experiments because they are easy to participate in and able to follow the experimental instructions reliably. Natural experiments are the kind of design at another extreme. In natural experiments, participants are in the daily-live conditions, and they do not know they are being observed.
Lab-in-the-field experimental research is located in a continuous spectrum between the "pure" lab experiments and natural experiments. It is conducted in a diversified environment with various types of subjects. The following four types of research experiments are distinguished according to the type of research problem they solve.
Specific populations
The hypothesis may have to be tested by using specific populations. Or the researcher hopes to test if a result could be generalized to a broader or a representative, population.
Measurement
Lab procedures can be put into use in the measurement of the field. Risk aversion, time preference (patience), altruism, cooperation, competition, and in-group discrimination are the most common measures.
Recruiting the treated
This kind of lab-in-the-field experiment is for the circumstance that participants who have already been treated are being recruited. The experimenters are recruiting rather than implementing treatments.
Teaching
The purpose of the experimenter may be to teach the target population about the games by using a lab-in-the-field approach. Through this process, the experimenter could target a particular policy problem. The game could be the treatment in this circumstance.
Audit study
Audit studies are often used for measuring bias or discrimination. Audit studies are part of the constructure of a giant field experiment. This field experiment aims to measure the behaviour of participants in the field rather than to change it. In most cases, researchers use the ways like sending messages to measure participants' behaviours unobtrusively.
Ethics
Experimental research methods in political science have unavoidable ethical questions: Can humans be used for experiments? While experiments in political science usually do not cause physical harm, the elements of deception is commonly existing.
When conducting political science experiments, researchers must intervene in the process of data generation. Political science, as social science, studies human behaviour. Then political science experiments will inevitably have an impact on people. For example, subjects in experiments make choices they would not otherwise face, or subjects are put into experiences or controls that they would not otherwise have.
The experimental process itself is not the only way political science research activities affect human life. Other influences include the influence of the dissemination of experimental results, the influence of political science scholars on their students, and the influence of research results on institutions and professional organizations. Just like the ethics of most other professions, political scientists have ethics.
In 1967, a committee was created by the American Political Science Association (APSA), which aims to explore issues “relevant to the problems of maintaining a high sense of professional standards and responsibilities.” Marver H. Bernstein served as the chairman of this committee. This committee created the first version of the written code of rules of professional conduct.
In 1968, Standing Committee on Professional Ethics was founded. Their job mainly includes the review of formal grievances, mediation and intermediation to other organizations, and issuing formal advisory opinions.
In 1989 and 2008, the code of rules of professional conduct was revised.
Ways ahead in experimental political science
After the 1970s, most political scholars began to use experimental methods to focus their research on political behaviour, public opinion, and mass communication. The classic topics include exploring the behavioural preferences and choices of different social groups in group actions, the influence of campaign propaganda on voting results in the voting process, the influence of media propaganda on public attitudes, and the influence of personality on political participation.
Since the beginning of the 21st century, political science research has clearly shown a trend from correlation research to causality research. Political scientists are increasingly dissatisfied with just confirming the strength of the relationship between various factors, and gradually devote themselves to the discussion of causal effects and mechanisms among variables. To measure causal effects accurately is a significant problem and challenge in the social sciences. The means to keep inferences from selection bias in observational studies are minimal. In experimental research, randomly assigning the observation objects to the experimental group and the control group can ensure that there are no observable or unobservable differences between the research objects in theory before the practical operation. The objectivity of the conclusion is built on randomization in this way.
The inherent flaws of various non-experimental study methods have been continuously exposed in the past 20 years. The most notable drawback among all is the limitation of observable data, which hinders researchers' further exploration of causality. In a modern society with highly developed information technology, it is becoming more and more efficient and low-cost to control and collect data by conducting experiments.
The emergence of new communication methods and new media is both a challenge and an opportunity for social science research. These social media and online blogs transmit a large amount of political information and make it more accessible to the public. In the past, the information for social science research was generally insufficient and deficient. Now it is necessary to face the problem of screening the validity of information in the era of information explosion.
Current political scientists are more and more inclined to explore political behaviour's psychological basis and attitude characteristics through experimental means. The distinctive feature of current research is that it focuses on some specific social groups, such as unmarried people, students, ethnic minorities, etc. Investigations about the general population are insufficient.
Different research methods have distinct advantages and disadvantages. The disadvantages of one method cannot be compared with the advantages of another method. More and more scholars have begun to use a mixture of different methods to make up for the shortcomings of a single experimental approach. In the field of social sciences, experimental research methods need to be combined with other research methods to obtain knowledge innovation in both theory and methodology and promote the development of the discipline.
