Prediction Markets for Science

vaniver

Prediction Markets for Science

vaniver

1 min readJan 2, 2023

Comments

Prediction Markets for Science — EA Forum

freedomandutility

There’s also this https://socialscienceprediction.org/ which is similar

Eva

Thanks for mentioning the Social Science Prediction Platform! We had some interest from other sciences as well.

With collaborators, we outlined some other reasons to forecast research results here: https://www.science.org/doi/10.1126/science.aaz1704. In short, forecasts can help to evaluate the novelty of a result (a double-edged sword: very unexpected results are more likely to be suspect), mitigate publication bias against null results / provide an alternative null, and over time help to improve the accuracy of forecasting. There are other reasons, as well, like identifying which treatment to test or which outcome variables to focus on (which might have the highest VoI). In the long run, if forecasts are linked to RCT results, it could also help us say more about those situations for which we don't have RCTs - but that's a longer-term goal. If this is an area of interest, I've got a podcast episode, EA Global presentation and some other things in this vein... this is probably the most detailed.

I agree that there's a lot of work in this area and decision makers actively interested in it. I'll also add that there's a lot of interest on the researcher side, which is key.

P.S. The SSPP is hiring web developers, if you know anyone who might be a good fit.

jacobpfau

In my opinion, the applications of prediction markets are much more general than these. I have a bunch of AI safety inspired markets up on Manifold and Metaculus. I'd say the main purpose of these markets is to direct future research and study. I'd phrase this use of markets as "A sub-field prioritization tool". The hope is that markets would help me integrate information such as (1) methodology's scalability e.g. in terms of data, compute, generalizability (2) research directions' rate of progress (3) diffusion of a given research direction through the rest of academia, and applications.

Here are a few more markets to give a sense of what other AI research-related markets are out there: Google Chatbot, $100M open-source model, retrieval in gpt-4

Michael Simm

I think this could be a very useful tool for improving public knowledge about the uncertainties in various areas of science.

But I wonder how can prediction markets be effectively implemented in decision-making processes.

For example, if there's a vaccine going through testing, or an intervention being studied, do you think there could be a smart way to integrate prediction markets into more optimal policy decisions?

You could also go the other way and call out interventions, policies, and other currently implemented things based on uncertainty in the prediction markets.