For a number of years, I worked for a research center in the United States, and participated in its annual research funding competition. The organization I was with was one of those you might call a “think tank” focused on independent advice and research. I was fortunate to participate, pitch & get funded several research projects, and it taught me a lot about how knowledge is created, and how people value it (or don’t).
All research and knowledge work, I found, exists on a spectrum from pure to applied.
Pure research is what people think of with academia. Researchers working on fundamental new techniques, or break-throughs. It tends to be very forward looking. It also tends to be very risky and uncertain, in that we know in science & experimentation that there are many blind alleys. Researchers don’t usually want to admit it, but the truth is that most pure research results in nothing in the long run. But occasionally, it produces breakthroughs.
Applied research is more concrete, and generally develops new techniques for specific problem sets. Certain kinds of startups operate in this mode, and most of our research did as well. In applied research you should be starting with real problems (not abstract ideas), but at the same time not just making incremental tweaks on existing methods.
Finally — knowledge work as a category is just any kind of work that requires the application of learning & knowledge. Most office jobs fall into the knowledge work category. And I think it’s useful to think of knowledge work as hyper-applied research. You apply existing tools & techniques to solve very concrete real problems, but often they’re operationally-focused problems the company hasn’t solved before. We use our brains and not our hands to produce value. We are usually not doing fundamentally new things, but nonetheless creating valuable outputs.
Circles of Knowledge
People’s acquisition of knowledge begins in primary school, where we are taught a broad range of basic facts, spanning many different areas (Math, Science, History, and so on). This continues in both secondary school and in the university, the general “broadening” of knowledge. The university starts to focus us in a particular topic area, but still has general knowledge pre-requisites.
At the graduate level, things get interesting: typically there are no longer any general knowledge pre-requisites, the study is all focused in one topic direction. Now we are deepening knowledge, instead of broadening.
If we think of the dashed line as the boundary of all human knowledge, then original research (both applied and pure) is pushing past that boundary and discovering or inventing new knowledge that hasn’t existed before. But notice how each area gets smaller as you go. Pushing out the limits is hard and effortful. An entire career worth of research may amount to a small extra “bud” off of the main circle of your domain’s knowledge.
Research pushes outside of the dotted line. Knowledge work stays strictly inside of the dotted line.
This is what an individual career & work amounts to in the scope of things. Collectively, if we have many thousands of researchers pushing the boundary though, we may expand the overall circle of human knowledge quite rapidly.
Knowledge as an Investment
Knowledge is its own reward, but its acquisition is costly, and in the real world we have bills to pay. When people join together into corporations, universities, and governments; those institutions have to be supported, and so we typically do it for a return. Through many decades, people have thought of knowledge work in terms of its investment and return characteristics.
Above is a breakdown of how I’ve come to think about it. These three categories of pure, applied, and knowledge work form a spectrum. In a sense they are all “made of the same stuff”, but they represent different risk profiles and desired results.
Is Research Worth It?
My experience in the research world produced mixed results on this question.
Of course pure research is worthwhile. If you want the fruits that research can produce, such as fundamental advances and new techniques — there is no alternative! No amount of incremental knowledge work will get you there. Research is about pushing the boundaries and not working smartly within them.
And of course knowledge work is worthwhile. We have real problems, day by day, that need solving now.
But “worth it” requires that we consider budgets and timelines, which are very real things. Institutional willingness to invest, and ability to take advantage of the results are often very real constraints. It’s fine and well that we need consistent progress on basic science to advance society, but that does not magically invent the dollars or the will to make it happen.
And so what I witnessed was a swinging pendulum through time.
When the organization is feeling wealthy, optimistic, and thinking long term, they invest more in research and reap the benefits. When they are feeling poor, pessimistic, or thinking short term, they don’t.
In a nutshell, institutions are always changing their mind about whether research is worth it, and this swinging pendulum is certain to continue.
As the fortunes of the economy and the company went, so too went the prospects for research. As companies and businesses can never be protected from business cycles, recessions, and so on — so too the idea of a stable steady stream of funding felt impossible. The government typically steps in and funds a huge amount of research, but this is also subject to external pressures like recessions, budget concerns, and even social debate about what kinds of research are proper, and so the pendulum swings with government, even outside of standard business cycles.
Conclusions
Short Term vs. Long Term Benefits: where to invest in the creation of knowledge is a pretty standard short-term vs. long-term tradeoff. So the answer boils down to that we need both, in balance. We need to increase our productivity for the future with basic research, and we need to produce today with knowledge work.
There is no optimal: finding the optimal allocation or investment is a trade-off game that is done in light of external pressures like recessions and wars. If there were an optimal allocation at a given moment in time, it would be sure to change as the changing world pulls our assumptions out from underneath of us as time moves on.
Forthright 