Collaboration

A goal of the Deliberation Lab is to establish a collaborative data collection and analysis ecosystem that allows researchers from across the Social and Computer Sciences to contribute to our collective understanding of small group deliberation.

Experiment-as-a-service

We are currently developing a researcher portal to enable external collaborators to leverage our virtual laboratory for their own deliberation experiments. Whether it’s forking existing experiment designs or creating entirely new experiments, researchers will have the flexibility to tailor designs to their specific research questions.

One of the unique features of the researcher portal will be its ability to guide researchers towards high-value experimental targets. Drawing on cumulative knowledge of the design space, the portal will identify areas that are undersampled or where conflicting findings exist. By suggesting these “interesting” regions, the portal helps coordinate the efforts of different research teams to advance the knowledge of the field as a whole.

The researcher portal will offer a flexible “experiment as a service” design that caters to the needs of different research teams. Teams have the option to bring their own participants and manage the entire recruitment, scheduling, and participant payment process independently. Alternatively, they can leverage the platform’s tools to streamline participant management, alleviating the burden of setting up their own infrastructure.

Shared dataset

Experiments run in the Deliberation Lab are designed to produce data in a standardized and well-documented format. This emphasis on standardization is crucial as it enables diverse research teams to contribute to a shared community dataset, fostering collaboration and facilitating meaningful comparisons between studies.

By having a standardized dataset, researchers from different teams can perform meta-analyses to explore how deliberations vary across different regions of the design space, uncovering patterns, trends, and insights that may not be apparent when studying individual experiments in isolation. This collective approach to data analysis and synthesis will help the field build cumulative knowledge and develop a deeper understanding of the contextual nuances of small group deliberation.

Prediction competitions

To understand its growing body of data, the deliberation lab will engage participants from the machine learning and social science communities in a series of prediction competitions.

Participants in the prediction competitions are presented with a ‘training’ dataset consisting of the publicly released experimental data collected to date, and a set of experimental conditions for experiments that have yet to be conducted or whose data has not yet been released. Participants then develop predictive models based on this data, on social scientific theory, or their own intuitions, and make predictions as to the result of these new experiment samples.

These competitions encourage researchers to explore innovative approaches, employ advanced analytical techniques, and leverage their domain knowledge to develop effective prediction models. Participants can experiment with different algorithms, feature engineering strategies, and modeling techniques to optimize their predictions.

The Deliberation Lab aims will recognize and reward the top-performing participants in the prediction competitions, showcasing their achievements and contributions to the study of small group deliberation. By comparing the predictive accuracy of unconstrained machine-learning models, theoretically informed formal models, and human judgement, we can estimate the contribution of theory and expert judgement in understanding deliberation outcomes.