By Nicholas Guttenberg
When I was at the Earth-Life Science Institute in Tokyo, one of the things I learned about was the difficulty of matching up the expected inputs and outputs of research projects in different fields so that those projects could work together. A researcher may work very hard to measure properties or estimate values that are considered central to the questions of their field, only to find that their potential collaborators can’t use that number because their questions operate at a different level of abstraction. Even within disciplines, one of the places that research often falls flat is when it comes time to convince other people that a hard-won result is at all relevant to them. One thing academic research projects and startup companies have in common that they’re often trying to create a space for what they can provide to be meaningful. Academics often tend to pick a thing of interest first, and then have to write papers and give talks that do as much work to sell their question as to establish the validity of the results. Startups may often begin with a founder’s ideas or expertise, but have to find a space in the market where they can provide something that people are willing to pay for.
The result of this can be that while the process of completing a single step of research and answering a single question can be pretty consistent, once you start to try to chain those research results together into a larger whole, things become quite a bit more challenging. If talks and papers thoroughly try to close out what they work on, should it be any surprise when they don’t in turn inspire researchers to change their own research directions?
What I want to explore in this post is if maybe there are things we can do to make this job of synthesis easier, and increase the degree to which other peoples’ work can impact our own process and the degree to which we can do work that impacts others. The focus here isn’t on general impact on the world, but specifically on research synergies. How should I interact with colleagues so that their research can be more successful? And how should I interact so that their research ends up being more likely to be something that in turn is going to impact my own?
I’ve found that the true value in attending large research conferences isn’t the talks, but rather it’s the stolen moments in the hallways or over meals where you can directly talk to someone about ideas, problems, or possibilities. Sessions are often made of short talks – 12 minute blitzes that mostly serve to advertise ‘we’re working on this thing, come talk to me elsewhere if you really want to understand it’ – or keynotes where a researcher summarizes and synthesizes the thing they’re best known for. But if you want to think about going somewhere new, usually you need a few hours with someone to get through the obligatory ‘I’m X and I did work Y on Z’ handshaking process.
On the other hand, there are a number of smaller workshops and conferences that really do try to work differently. Gordon Research Conferences are well known for encouraging people to talk about in-progress unpublished work rather than stuff that is already done and frozen, meaning that there’s still some chance that interactions could actually do something to change the trajectory of the work being presented. Rather than quick question and answer sessions, the same amount of time is given to discussion as to the talks. The schedule is such that everyone at the conference can see every talk, and large amounts of time are provided around lunch so that people have the opportunity to discuss what they just heard with each-other, seek out the speaker, etc.
A Cross Labs workshop on open-endedness I attended recently organized by Lisa Soros used a format based on the Dagstuhl Seminars, and strongly encouraged the participants to actively pose questions to each-other and break out into small groups for discussion from the start. While there were invited talks (occupying about one third of the total workshop time), by and large the talks were provocative in nature, again posing questions and sharing possible approaches rather than reporting on work that the speaker had done. As a result, it felt like people left with new ideas for research projects for which there was a guaranteed audience of fellow researchers who had already established their interest.
These kinds of patterns strike me as having a key property with regards to making it easier for us to benefit from each-others research: people are either indirectly or directly sharing what sorts of things would have an impact on them and their research.
There are a couple of different elements to this to unpack. I’ll take the perspective that there is some grand challenge or big problem with lots of uncertainty, lots of conflicting definitions and lack of clarity, and where no single researcher is going to fit ‘the solution’ into one talk, and we want to figure out how to organize interactions between researchers in that community in the context of workshops, meetings, reports, etc. In the spirit of this post, I won’t present these as things I have all the answers for, but rather as things I would like to know how to advance as the values and norms of a new kind of research culture.
1. Establishing interest and motivations
Often research starts with some local source of interest or need. The solution to a technical problem hints at some broader principles, or something noticed while tuning an experiment suggests potential for a deeper understanding. But then, when it comes time to present those results to the broader community it can be very hard to bridge the gap with an unknown audience such as a funding opportunity or journal or conference. It’s not because the research is inherently irrelevant, but because it may well be that the people who would care aren’t in that particular audience.
So finding those potential connections ahead of time rather than waiting until the research is finished could be really valuable. This could involve making small adjustments to the research to tune it to a particular community as the project proceeds, choosing research directly based on an interest someone else in the community expresses, or just changing the way things are framed or interpreted to make connections.
For example, the Cross Labs open-endedness workshop opened with asking each participant to write down a question they thought would be good to answer over the course of the week, share those in groups, and then had those groups summarize the take-aways. This was done first thing, before any talks or presentations.
Thus we get an idea ahead of time what questions others have, and we can start evaluating what abilities we have to address those questions.
My ambition here would be to discourage the habit of bringing or expecting prepared material, and encouraging things to be made in response to the interactions even if it means that things are less polished. In the extreme, not a slide should be made without hearing this from the other people at the table first. That way when we do inevitably lapse into talking about our own research we can be specific and pick the bits that are actively relevant, rather than giving a sales pitch for all the over-and-done stuff.
2. Saying where we’re stuck or confused
There’s a bias towards reporting successes over exposing weaknesses. In reputation-driven career paths, every talk is a chance to show cleverness and be hired or to make a good impression on a colleague who will then send a letter to a tenure review board or review a grant in the future. By instead saying ‘I couldn’t do this’ or ‘we’re stuck here and we don’t know how to get past it’ we create potential inspirations for other researchers. We show that here’s a problem we know at least one other person would care about, and which would help them advance their research as well.
I also wonder if there’s a reluctance to give away good potential research directions or opportunities. If you encounter a particular difficulty that you yourself could eventually solve and which other people might have too, that’s a publication right there. Sharing the difficulty could mean giving up that opportunity. At a system level, if people benefited more from the overall progress of a field or endeavor than being able to take credit for the particular movements, it seems like it might help this. However, at the level of a workshop or small research community, it might be necessary to work around the realities of the system we currently have and figure out how to get people to open up without feeling like it’s self-sabotage.
3. Agreeing on what would constitute a solution or impactful result
Beyond identifying our own weaknesses, failures, and sticking points I think it’s also important to pre-commit in some sense to what would constitute a solution. This serves a subtly different purpose. Saying where we’re stuck creates a prompt for synergistic research to be done, but establishing what constitutes or constrains possible solutions prevents that research from being wasted when there’s some unspoken mismatch in expectations. This is especially important where things get murky and a lot of the work isn’t about solving well-defined technical problems, but rather finding ways to think clearly about vague ideas. Hashing out what a solution would entail prevents the frustration of constantly moving goal-posts, and can expose places where a lack of clarity is causing problems before people go away and invest a lot of time in things.
So why now? GoodAI grants
GoodAI has now given out a number of non-traditional grants to researchers and research groups. Among the unusual aspects of these is that the grants are intended to produce research that GoodAI itself will directly incorporate into its own research. So far we’ve had introductory meetings where recipients presented their research to the GoodAI team, but in the future, we want to try to organize ways for GoodAI researchers to work along-side or in parallel with the recipients, and ultimately to form a research community in which the recipients of the grants can work together, benefit from each other’s expertise, and influence the trajectories of each-other’s projects. To do that, we need to think about ways to get out of the usual ‘I will tell you what I am going to do, I will do it, I will tell you what I did’ kinds of structures of interactions. Rather than reports, we could have workshops, moderated panel discussions, or other things centered around not just providing an update on status but actually exposing the details of the research process. That way, not only would the different pieces of research be more compatible when it comes time to synthesize them, but we would also have a better chance of being able to transfer the implicit knowledge that is built up during a project but which doesn’t make it into papers and talks.
Let us know what you think in the comments section below!
Leave a comment