Interview with Professor Doug Altman

The interview covers questions about his career, research interests, thoughts on the current research climate and advices for young research fellows.

The transcript is available below.

Did you know from the very beginning that you wanted to be a researcher in the field of biostatistics? What was it that sparked your interest in that field?

I did a first degree in Statistics, so I’m not really a mathematician, it was the applied element of the subject that always appealed to me. I had a huge stroke of luck in getting my first job after university in what we would now call a Department of Clinical Epidemiology working on large epidemiological studies. I knew really from the moment I went there that it was what I wanted to do. That was in the 1970’s and I have to say I have never regretted my choice over the entire 45 years or so of my career. I was also very fortunate that I was a member of a department that received a lot of overseas sabbatical visitors. I had a fantastic opportunity to hear and be influenced by these visiting academics, including David Sackett in particular, who spent a year at the department. I only have my first degree. After that I just worked. I don’t have a masters or a PhD. I do have a honorary doctorate from the University of London.

Would you say that meeting these prominent academics at that time motivated and influenced you to keep moving forward in this field of research?

I think sub-consciously yes, with the benefit of hindsight looking back on what influences I had. But that influence was on doing good clinical research. Indeed, for the first twenty years of my career I worked only on clinical data, I didn’t do any methodological work at all. I was working as a medical statistician on numerous medical projects. My 1990 textbook includes a large number of datasets which came from the collaborative projects I had been working on. Since then my focus has been much more on methodological studies although I have continued to do some clinical studies, notably in fetal medicine, cancer, surgery, and colorectal screening.

What were the most difficult things you had to face during your career?

I spent six years working at the Department of Clinical Epidemiology and then moved to the Medical Research Council where I spent eleven years working on a huge number of clinical projects. One of the big problems I encountered in those days was that people didn’t have protocols and they didn’t ask me to help them design the study. They would do the study and then come along with data and then ask for my help. The challenge was in the first instance to figure what the research question was and then perhaps to realise that the study they had done wasn’t appropriate to the question they thought they were answering. It was often very difficult to decide if it was feasible to rescue a bad project. It was also difficult to tell people that what they had done was actually a waste of time and if only they had come to you first they could have done something really good; it isn’t an easy thing to do. It also means that analysis of such projects was often far too complicated as the studies weren’t designed properly. However, what we have in the current world is totally different. There has been a massive change in the way statisticians are involved in clinical research and in particular in clinical trials. It is completely impossible, here in the UK at least, to do a trial now without having statisticians and probably other methodologists involved in the planning of that work.

What has been your proudest career moment to date?

The thing I’m most proud of achieving is writing a textbook. It is extremely easy to start one, but extremely difficult to finish. I wrote it all in my spare time, I didn’t do it at work; I did it at the evenings and weekends over many years. I think finishing it was great achievement and I’m very proud of that. I read a lot and I learned a lot during the process. That was twenty-five years ago, but people are still buying it, which is really remarkable. When you write a book like that, you want people to find it helpful and the positive feedback I have received about it has been very gratifying.

What inspired you to found the EQUATOR network?

It was the culmination of quite a long process. In the late 1980’s I did a study with my colleague [Caroline Doré] where we looked at randomized trials published in the top medical journals. We found a lot of deficiencies in the reporting and we published that in the Lancet in the 1990. Quite a lot of what was published in that paper echoes what would happen a few years later with the development of the CONSORT statement. I wasn’t involved in the development of the statement, although I peer reviewed it for JAMA in 1996. In 1997, I was invited to join that group and was subsequently involved in other reporting guidelines. By the mid-2000s it became apparent that although people were producing guidelines they wasn’t having the impact that we wanted. You realise that the standard publication approach is very passive; you write something and hope it will be published and that people will read it and be influenced by it. It became clear that the impact of reporting guidelines was detectable but much lower than what we wanted. So EQUATOR grew out of the recognition that we needed to move from being entirely passive to being active and proactive. To try to make things happen, to draw people’s attention to the existence of the reporting guidelines. To try to get changes in education and journal policy and so on, while continually doing research to support that need. We become more aware of the need for this and it was something I discussed with David Moher, in particular in relation to CONSORT. I managed to get a small grant here in Oxford which proved hugely beneficial in moving things along, we were able to employ someone for a year to get the EQUATOR agenda started, and that person [Iveta Simera] is still here ten years later thanks to further funding. So the initial impetus was recognising that the people who develop guidelines don’t necessarily have time to be promoting them, so we needed a different type of initiative to improve recognition of importance of good reporting and adherence to reporting guidelines. The biggest indicator of what we have done is on our website.

How satisfied are you with the impact of EQUATOR on health research compared to what you hoped for when the network was created?

I’m satisfied that we are still going. Recognition of this type of initiative takes five to ten years – that is recognition before you get the impact. I think you have made a mark when people talk about your initiative increasingly and assume that other people know what it is. In that sense we are making good progress. We now have several full time employees here in Oxford as well as four centres around the world. I don’t think we are making waves, but we are making ripples. So it is satisfying, but we are nowhere near satisfied. We are always looking to do more.

What are the main challenges that EQUATOR should tackle in the future?

There are some massive problems with the current research climate. There is a huge increase in the amounts of research taking place. There is increased pressure on people to publish. There are all sorts of reasons why people are doing research less well than they should do and reporting it less well than they should do. It is easy for a journal to say we want you to adhere to CONSORT or STROBE etc., but it is a lot of work to ensure that authors actually do that, even those publishing in the top medical journals. Of course we also don’t want a case of “good reporting and bad research”. EQUATOR recognises that and we are trying to extend our scope beyond reporting and also include good research conduct and planning. The more planning you do, the better the research is. The difficulty is getting to people at the right time-point, for example when they are writing the paper, because when they are submitting their paper it is too late.

Are you optimistic or worried about the evolution of clinical research? Why?

I am worried and optimistic. I think the challenges we face are major. They are reflected in two big problems that we have. One is the selective publication of research findings and the (mis)interpretation of those findings – those two things go hand in hand. The second area of concern is the continuing dominance of statistical significance as a mark of truth, which is nonsense. People have been pointing out this issue for the past sixty years. How optimistic can you be when people have to point out the same problem for sixty years? People have been complaining about bad research for a century in medical journals. Unfortunately it is not necessarily in people’s best interest to do high quality research if their goal is to get a lot of publications. Unfortunately, in modern times researchers’ livelihoods depend on continuing to get more grants, and if you show your bright idea doesn’t work, you are not going to get any more grants. So you have a lot of incentive to distort the findings to make it look like a promising project and therefore get more grants. It is a global problem and it is a systemic problem. It is not within any one organisation’s power to make a major change. There are a lot of reasons out there why we should not be too optimistic.

What do you think has given rise to the current lack of reproducibility in research findings?

There are current concerns about the lack of reproducibility of research findings. There isn’t even consensus on what we mean by reproducibility. There is a lot of confusing literature on what people think reproducibility is, because it has multiple meanings. The fact is that in many fields, this is a major concern. Non-reproducible findings are in part a consequence of selective publication and the dominance of significance as a means of drawing inference.

So all of these concerns and initiatives led to your interest in joining the MiRoR project?

I think the MiRoR project is very exciting for various reasons. If you look at the quality of the whole team engaged in it, we all see that this is massively important. I think part of the way forward is to get good research practises out there into early career training before people develop bad ideas.

What foundations do you believe are essential to preserve the integrity of the medical research field moving forward? What lesson can be learned from the past?

Well, I do detect a lack of realisation that what really matters is clear thinking rather than being very skilled in certain techniques. The major problems out there aren’t usually about people using the wrong methods, they are more conceptual, more about choosing an inappropriate design. So nothing to do with some fancy modelling, problems are more conceptual. They relate to the appropriateness of a particular design or data set to address the question that one is interested in. I think another feature of the over-emphasis on statistical analysis is that people forget that what really matters is having to trust the data, that the quality of the data is hugely important. It’s no good applying fancy complex methods to lousy data and unfortunately the new move to using so called “big data”, means that you are trading quantity for quality. So you get much more data, but it’s less high quality intrinsically in terms of the quality of the measurements and also in terms of the missing data. The fact that measurements are not taken to a standardised protocol, leads to inconsistency across places. Being aware of all of these issues is really important, knowing how to handle them is tough, but if you don’t know that these are problems you won’t even think about them.

So I would say that people need to be aware of the whole context, the quality of the data, the appropriateness of the data set to address a particular question. These problems are particularly acute for studies using existing data. They shouldn’t be a problem in a prospective study, but they can be. And then I’d emphasize understanding what statistical analysis can and can’t do in the way of statistical inference. The last thing I would say is the classic thing that you should interpret a study finding in the light of what else is already known, most obviously in systematic review, but it’s remarkable how often people interpret their study as if it were the only one ever done.

What would you say is the most important paper/ book you think we should all read as early stage researchers?

Well I think any book that taps into the thoughts I gave you. Anything in relation in how to think statistically. I wouldn’t particularly recommend it now, but one I wish I had been exposed to when I started my career is by Donald Mainland in which he emphasised the importance of statistical thinking over statistical arithmetic, I think that is such an important idea.

There are some good books. Most of them try to cover everything from study design to data analysis inference including my own. I’m not sure I can recommend a particular book. Something that is more focussed on getting your ideas straight on how to think conceptually about doing research. Basically research starts with a question, for any given question there are a couple ways in which you can go, in using existing data or getting new data and perhaps it’s not easy to find good guidance on that. I’m actually involved in a group that is trying to develop some guidance on that for observational studies. There are loads of books on analysis – I don’t think that is the issue, I think there is a need for guidance on statistical thinking and research design.

What do you think we can do to attract more young researchers into the field of biostatistics?

Well obviously the number you can attract depends on how many positions there are. I think it’s very interesting, once people are in biostatistics they find it very exciting, and very few leave to go into other careers. Yet a lot of people find their way into biostatistics after being educated in other scientific areas, that has always been the case, it may be less common now but certainly some of the best known statisticians of my generation, and maybe also the next one, started off in other scientific areas. Maybe the reason is people have a false idea as to what statistics is. It is still seen as number crunching and about analysing data. Actually when I give talks to students I sometimes ask them “Well what you think a medical statistician does?” The last time I did this I had a lot of suggestions but none of them mentioned the idea of helping people design good quality research, so I think unfortunately we are stuck with the term statistics or biostatistics, it doesn’t convey to people that we help people design good quality research studies to answer important questions or something like that. Most of the time of the statistician is spent in designing research not in analysing it, and that truth is actually rather well hidden to the outside world. I think it is much more challenging and satisfying to be involved right from the outset of a project and help to go right through from the question to the publication if you can manage to do that, the whole life cycle. It’s quite hard with clinical trials, because of their duration, but it is possible with smaller studies.

Finally what would you advise all the fellows who are part of the MiRoR project in their future research careers?

I think that there is a danger of being too narrow in your reading and activities and I encourage people, even if you are working on one quite narrow topic, to try taking a broader interest. Try to be aware of broader issues in terms of general scientific conduct. You will get that through the MiRoR project because there is a lot of emphasis on the research climate. I think if you are working as a statistician, for example in cancer research, you are probably working on a very small number of statistical techniques in terms of question, design, and analysis. You can become an expert in these but not evolve broadly as a statistician; the same applies for all researchers. So keep a broad interest and go to talks that aren’t directly relevant because you might learn something. I was fortunate I was able to do that when I was starting out as I was in walking distance of the Royal Statistical Society meetings, so I would go to those very often. The internet is fantastic – as you know, you can waste millions of hours on the internet and read all sorts of stuff instead of doing what you should be doing. I think reading the general journals, the BMJ, JAMA and journals like that, is important. Be aware of the wider field of medical research and not just the topic you are working on.

Thank you so much for taking the time to speak to us today. We really appreciate it, you have given us a lot to think about and learn from. I think we can speak for all the PhD students when we say everybody is very much looking forward to seeing this interview. Is there anything else you would like to add before we leave?

Take opportunities that arise. I got to where I am by seeing opportunities to try something and just going with it. I think this is the sort of programme I would have loved to have been on so I’m very pleased to be part of it and looking forward to having some of you come and spend a bit of time with us here in Oxford, and also I look forward to whenever we all meet again.

À lire aussi