AI general models and the future (?) of bioinformatics research

I finally got around to trying Claude code on a simple "discovery" bioinformatics project. As we often do in the group, the idea was to gather some existing public data and try to combine it in a somewhat new way. The topic was cross linking MS and protein structures but I want to focus more on the process of doing the work with the AI model rather than the scientific example itself. For context, I haven’t tried to actively do any bioinformatics coding and research myself in over 10 years, so this was also quite fun to try out.  

In two separate sessions I used Claude code in two different ways - a more guided approach where I instructed the data exploration, asking only sometimes for suggestions on what to do next; and a hands-free approach where I got the model to make a plan and then let it process it until completion. In both cases, the objective was to start from the idea and finish in a fully written draft manuscript. The models used were also not the same, using a lower accuracy model (Sonnet 4.6) in the guided test and the latest release (Opus 4.8) in the test where the model went from plan to manuscript by itself. 

The project was not very ambitious to begin with, but in both cases, I think the end result was sufficiently interesting that it could have been expanded towards an actual study and paper, with all the caveats that the outputs of these models are not trustworthy. I was still very impressed with the general capabilities to set-up the needed tools; look-up papers and attempt to download supplementary files; find the correct API calls to databases; then to generate analysis code, run the analyses and make plots and figures. For example, it was straightforward to get the AI models to use the newly released AlphaFold homo and hetero complex structures from the AlphaFold database. There were of course issues. There are many websites that are blocking access; it made up the reference to one of the studies that did exist but under another reference; it needed guidance for some outputs (e.g. rendering nice looking structures); the figures in general look really poor in taste. I started the project with a clear but vague direction of taking cross linking MS data, collected from different studies, and combining it with protein structural models. From this vague direction I tried to get the model to suggest some “novel” analyses. Predictably, the ideas were very incremental even if they were still useful.   

Step-by-step interactions vs automated research

Guiding the model step-by-step felt a lot more like the standard way one would do this kind of research without the AI models. Generating scripts to combine datasets and doing quality control analysis on them. Combining different types of data in a first analysis which may trigger a new idea and so on. This can work well while the AI model keeps most of the project progress in its context window and can make connections between analyses done along the way. The key difference to regular bioinformatics research is the speed of writing the code and dealing with IT issues, figure plotting, etc. It really just boils it down to the research, looking at plots, asking questions, considering what to do next. The obvious drawback is that the code generated could be filled with errors of course. Some of this can surface by intuition during QC when the plots show unexpected behaviours but ultimately any of this could only be used after going through the code carefully.

The second time around, I just told the model the general direction, and asked it to make a plan for research that would go from data analysis to manuscript on the same topic. I mostly just agreed with the plan and let it run for 1h until it produced an output. I looked at the manuscript and made some suggestions, including an expansion in one of the analyses. This was less fun I would say. It was more like giving some feedback on someone else’s project. One big advantage of this approach was that the output was very well organized in a reproducible research sort of way with data, code, figures and text. The step-by-step guided study generated a sprawl of code and figures that would be harder for anyone to pick up.  This second version seemed to pay more attention to more details but this is likely more due to the fact that I used a more powerful model as well. 

The figure below is an example output from such an attempt, purely to illustrate the complexity/simplicity of what I am discussing for those that may recognize what is described in the figures. None of this has been validated so I don’t mean to suggest that this is scientifically correct. Assuming the code was correct, it would actually make for a very modest and very incremental, but likely publishable bit of science.

Rapid prototyping as a strong use-case

So how useful is this in the end ? As it stands, the code and produced analyses need to be verified by experts. It has been too many years for me to compare against how fast I would be able to do if I wanted to do this same work without AI. It would certainly take me more than 1h, that is for sure. My impression is it should be an increase in productivity even for an expert that needs to verify the code, in particular when starting up something new. It would be ideal for non coders but the output can’t be trusted. 

This “experiment” does not touch upon bioinformatic research that is automatically verifiable. For example, I may want to build a computational method that predicts protein-protein interactions based on a specific type of data. If the problem is set-up correctly, then the AI model could be tasked with trying a wide array of different methods until it finds the best solution. In this case, the validity of the solution can be automatically checked. This is, in any way, what conceptually has been happening with deep learning method development. 

As it stands, I think the best use case I can imagine for “discovery” bioinformatics is in rapid prototyping of research ideas. This can derisk such efforts and make it easy to explore very quickly a research direction of the sort I am explaining here. The key is that the methods used need to ideally be within distribution. This is like saying that the types of methods need to be in the training data. This does not mean that the outputs cannot be novel research. There is a lot of bioinformatics research that basically consists of applying well known methods to some different combinations of datasets. If we ever reach a point where these outputs can be trusted without expert validation then this could really be a big boost to science. I am not worried about the future of bioinformatics. This would actually mean that bioinformatics would explode in usage given the unmet need there is for more data analysis in biology. 

For me, the interactive approach brought back that fun experience of just playing with data. If I had the time I could certainly imagine trying to test how multiple such data and idea explorations could be done in parallel. I think scientists with broad perspectives, generalists, will have good opportunities in this new world.