Teaching basic lab skills
for research computing


MIT 2013

Software Carpentry training has had a tremendous influence on the productivity of the students in my lab and several other labs on campus. It trained them to write code that works properly more quickly and to automate repetitive tasks that they used to spend hours on (now they spend minutes). In addition, we are now reusing code much more often, because the code is written in such a way that the students can remember what they did and other students can understand exactly how the code works. I can't recommend Software Carpentry strongly enough.

Ethan White is an Associate Professor in the Department of Biology and the Ecology Center at Utah State University.

My labmates and I attended a Software Carpentry workshop a few months ago, and since then, several graduate students and RAs have been much more willing to tackle simple programming tasks independently. We have also now implemented a version control repository in the lab: it has greatly reduced the number of files flying around and helped to ensure that everyone in the lab has the most up-to-date versions of all the code we routinely use. Its history mechanism also keeps us from having to manually document every change made or analysis done, which has greatly improved our ability to find information quickly.

Lynne Williams studies the cognitive neuroscience of language development and develops statistical techniques to analyze large multivariate data sets.

As a nuclear engineer at the US national labs, the number of nuclear safety and performance codes I see which are practically untested, basically undocumented, and completely lacking in version control is alarming. New projects are starting to incorporate better practices, but over and over again, students and new staff members show up ready to implement advanced numerical methods but don't have a handle on version control. I've found that the time it takes to train new researchers to adopt essential software practices is vastly reduced by the workshops and online resources provided by Software Carpentry. By directing colleagues to the online lectures instead of teaching them individually to test and version control their code I've personally saved a great number of hours and they've been spared just as many false starts.

Katy Huff is a post-doc in nuclear engineering at the University of California, Berkeley.

As a self-taught programmer, Software Carpentry gave me the tools to plan better code before even picking up a keyboard—improving my ability to expand, re-use, and test what I develop. The result is a more reliable, better traceable product, and I have a better sense of what to look for when evaluating other tools.

Deanna Langer's PhD explored multi-parametric MRI for prostate cancer localization

Armed with a single introductory C++ course, I did a master's degree and several years of consulting work on spatial simulation models before taking Software Carpentry. Long, slow, frustrating experience left me well prepared to appreciate this course. What has changed? I work more quickly, and re-use my own code; I find more errors, and spend less time fixing them; I trust my results more; I don't mind revisiting and revising old work; collaborators and potential employers are more impressed; and I'm happier.

Josie Hughes develops models of mountain pine beetles and other outbreaking forest insects

I started working at the Space Science Lab last November focusing on QA for two satellite projects' flight software. While I have written and tested lots of software in the past, I was unfamiliar with Python which is the preferred language for software testing here at the lab. The workshop gave me the tools to start to feel competent in my new job.

Irene Rosen helps test satellite flight software at UC Berkeley

My involvement with Software Carpentry as an workshop instructor and lesson contributor has allowed me to pursue new career paths; most notably, my current position at the University of Wisconsin, Madison. As a Software Carpentry instructor, I gained valuable cross-discipline experience: communicating difficult concepts to non-experts, developing skills in computing tools, collaborating with a team, and demonstrating my commitment to better computing skills by volunteering my time. My involvement with Software Carpentry also introduced me to a unique professional community of people who care about research and computing, which is typically not found in a single university/lab/office. This community has introduced me to new ideas, tools, and job postings from off the beaten path.

Christina Koch helps researchers at the University of Wisconsin, Madison, access and utilize large-scale computing resources to transform their research.

Software Carpentry helped me to explore the ways in which I improve and maintain my code. This made it easier for me to look back upon my old work and adapt it for new purposes, thereby making my research progress more quickly. It has also let me ask types of questions that I previously had no ability to address.

Minyoung Wyman studies the evolution of male and female differences using fruit flies as a model organism

I depend on the techniques taught in this course to maintain reliable programs, scripts and data organization. The time management, revision control, debugging and testing strategies are absolutely essential to build and maintain my projects. The basic Unix, shell, and scripting skills save me significant time and effort every day. It's an invaluable resource for computational researchers.

Anita Oder is helping develop a set of optimal experimental planning and analysis tools for neuroimaging research

While doing Software Carpentry, I made significant changes to the program I was using in my research. I felt like I was finally in control of its behavior, not vice versa. I still refer Software Carpentry to anyone new to programming, best practices, or even Python, especially if I have to work with that person.

Elango Cheran's MSc thesis was on detecting structural variants from paired-end sequencing data

Like most psychology students, I got no formal training in computational methods during my undergrad or graduate program. This meant that I was effectively cut off from the tool I was using and pessimistic about my chances of ever mastering it. Software Carpentry demystified programming for me and also gave me a solid background in software development practices. I now use tools that I never would have considered before taking the course, and have even accepted a postdoc in a computation-heavy lab, which would have been out of the question. My only regret is that I wasn't able to take this course earlier in my degree!

Hanah Chapman studies the evolutionary psychology and cognitive neuroscience of human emotion

Deadline after deadline: research life is as simple as that. As a consequence, research software is too often written in a rush. How many of you have faced that little, tricky, hidden bug a few days before submission? If you did, you also know that feeling of mistrust of your code. Software Carpentry showed me how easy and rewarding it is to write code that is testable from very early stages: write a little function, and immediately check whether simple examples yield the results you expect. And while Subversion repositories are not so easy to set up, once they are working, it's like having a perfect lab assistant to tell you what is going on at any time.

Enzo De Sena builds physical and acoustic models of surround sound

Before I attended the Software Carpentry workshop, I honestly didn't think I'd pick up anything useful other than a few minor tips and tricks because I had been programming for over 6 years at that point. Much to my surprise, Software Carpentry transformed the way I approach programming. I learned how to write better, flexible software with command-line options; the importance of open-source, reusable software in research; and was introduced to the best research notebook I've encountered so far: IPython Notebook. I couldn't recommend Software Carpentry enough, even for the veteran programmers out there.

Randal S. Olson studies biologically-inspired artificial brains and algorithms at Michigan State University

I was a physicist in a biology lab, working mostly with biologist with little or no knowledge of programming. What Software Carpentry taught completely changed our way of working. It was hard to understand at first why seemingly simple tasks sometimes took longer, but after a few weeks we were able to automate days of work so that we could have a coffee break and brainstorm on the actual science. This completely changed the way we envisioned studies, from limiting ourself to what a student could manually analyze to collecting as much data as possible in order to investigate modification of mouse Oocytes maturation at timescales we wouldn't have looked at. And even though we didn't have deep training in some topics, just knowing that things like regular expressions existed allowed the team to consult experts. This allowed us to structure the way we did experiments and store data in better ways, which in turn made getting new team members up to speed much easier.

Matthias Bussonnier is a post-doc at the Berkeley Institute for Data Science

Have a story of your own you'd like to share? Please let us know.