Blog

Jenna Lang has posted a great wrap-up on the boot camp at UC Davis — with Python cookies!

Mario Antonioletti has posted his experiences on being a first-time instructor at our boot camp for the Oxford doctoral training centres, our second in Oxford, last week.

Hack4ac is a one-day hackathon in London, England, on July 6. Its goals are:

• Demonstrate the value of the CC-BY licence within academia. We are interested in supporting innovations around and on top of the literature.
• Reach out to academics who are keen to learn or improve their programming skills to better their research. We're especially interested in academics who have never coded before.

It looks like fun—if you're in the area and interested in getting involved, they'd be happy to have you there.

Vijee Venkatraman has written a good article for Science Careers titled "When All Science Becomes Data Science", which mentions Software Carpentry.

Software Carpentry's mission is to help scientists teach other scientists how to be better programmers. If we want to do that successfully, we need to be scientists ourselves. In particular, we need to base what we teach on evidence, not anecdotes or personal preferences.

For example: we taught Git at the Toronto boot camp last week, and once again I think our learners would have absorbed more if we'd taught Subversion. Why? Well, take a look at this diagram by Oliver Steele (which I found on this page written by Nick Quaranto):

Four locations are in play, and eight different commands are used to move information around or compare what's in one place to another. Now look at the corresponding diagram for Subversion:

There are only three locations for people to keep track of, and the basic workflow involves only four commands. If you believe complexity is partially multiplicative (because people have to keep track of the interactions between things, as well as the things themselves), Git is at least twice as complex for people to understand. Slicing it another way, there are more opportunities for people to do the wrong thing with Git, and more they have to understand to undo it. And that's before we introduce branches...

But that's just my point of view, and the fact that I can wrap a plausible story around it doesn't make it true. Matt Davis and others believe they've been successful with Git in front of the same kinds of people. We need to find out who's right, so our major goal for the next three months is to poll and interview boot camp attendees to find out who was taught what, who tried what, who's still using what, and why. Caitlyn Pickens will be leading this effort, and many of you will get email from her in the next few weeks. When it arrives, please take a few minutes to tell us what you're actually doing, even if it's not what we told you to do—especially if it's not, because that'll tell us what we need to fix.

The hosts of our February boot camp at the AMOS conference in Melbourne have collected some more detailed feedback from participants. I'm pleased that two thirds thought the content was just right, and even more pleased that 83% thought version control "must be taught".

How useful did you find the online software installation instructions?

How did you feel about the schedule?

The lunch and tea breaks were:

What do you think is the ideal length of time for a boot camp?

Rate the following topics in terms of how important you think they are for a weather/climate science audience.

In general, you found the content:

"Online office hours" is a service provided by Software Carpentry where you can get online assistance over a 2 hour period once a week. Which item best describes your current engagement with that service?

If a boot camp (of similar content) was held in conjunction with the AMOS conference in Hobart next February, do you think you would attend (i.e. as a refresher)?

Software Carpentry has focused on computing for most of its 14 years (primarily because that's what I'm most familiar with) but it's increasingly clear that we need to tackle other parts of the research cycle. One is the new ideas clustered around publication, discovery and metrics, which I'll discuss in a future post. The other is data management; we only touch on the topic right now, but it's as important to most scientists as crunching numbers, and how best to do it is changing rapidly. Luckily, a few of our friends have written a guide for the perplexed:

Ethan P. White, Elita Baldridge, Zachary T. Brym, Kenneth J. Locey, Daniel J. McGlinn, and Sarah R. Supp: "Nine simple ways to make it easier to (re)use your data". PeerJ PrePrints, v1 received 2013-04-11, DOI: 10.7287/peerj.preprints.7.

---

Sharing data is increasingly considered to be an important part of the scientific process. Making your data publicly available allows original results to be reproduced and new analyses to be conducted. While sharing your data is the first step in allowing reuse, it is also important that the data be easy understand and use. We describe nine simple ways to make it easy to reuse the data that you share and also make it easier to work with it yourself. Our recommendations focus on making your data understandable, easy to analyze, and readily available to the wider community of scientists.

Their nine specific recommendations (elaborated at readable length in the paper) are:

1. Share your data.
2. Provide metadata.
3. Provide an unprocessed form of the data.
4. Use standard data formats.
5. Use good null values.
6. Perform basic quality control.
7. Use an established repository.
8. Use an established and liberal license.

It's a great outline for a half-day introduction to data management as part of an "extended play" Software Carpentry course, particularly when combined with William Stafford Noble's "A Quick Guide to Organizing Computational Biology Projects". We hope to turn the pair into lessons by September.

Digital badges are a hot meme right now. They let anyone, anywhere, issue credentials that are finer-grained than degree certificates or driver's licenses. Want people to know that you can change the oil in a car? There's a badge for that. Or that you can speak conversational Frisian? There's a badge for that too. And "backpack" sites make it easy to aggregate badges, including the ones that show you've created content for Software Carpentry, that you're qualified to teach it, or that you've helped to organize and run a boot camp.

In discussion with Cameron Neylon earlier today, another use case for badges came up—one that will require a bit of tooling, but would be useful in a lot of contexts. When I submit a paper to a scientific journal, the reviews that come back are usually anonymous (i.e., I don't know who the reviewers were). There are good reasons for this, but it creates a problem: how do I know how to assess those reviews? To borrow Cameron's example, if my physics paper gets one review from Richard Feynman and one from Joe the Mechanic, I probably ought to pay more attention to Feynman's—unless the paper is describing an experimental setup, in which case I should probably care more about Joe's, because Feynman was notoriously bad at doing experiments.

I can't solve this problem with badges right now because each badge identifies the person it was issued for (which is kind of the point). But what if Jane, as a reviewer, could go back to the badge issue (or to the backpack site she's using to aggregate her badges) and say, "Please give me a token I can attach to this review to show that I have an Expert Experimentalist badge"? The token would be digitally signed, so that people could confirm its authenticity, but no personal identification.

By analogy with Peter Wayner's "translucent databases", we can think of these tokens as "translucent badges": they let some light through, but they're not completely transparent. I can see lots of other ways they'd be useful. For example, I would really like to know whether a lengthy comment on Slashdot about software patents was written by a patent lawyer or a teenager in a basement in Saskatchewan—except what I really mean is, "I'd like to know how much the commenter knows about the subject," because that kid in Saskatchewan just might be a self-taught expert. Badges give her a way to validate her expertise; translucency would give her a way to share it more safely.

Parnas and Clement's 1986 paper "A Rational Design Process: How and Why to Fake It" [1] is one of the most widely read in the history of software engineering. In it, they argued that designing software according to some particular process isn't what matters; what does is creating documentation after the fact to make it look as though a rational process was followed so that other people can retrace the designers' thinking without heroic effort. Acknowledging the messiness of reality wasn't new: a century ago, Poincaré wrote that most mathematicians figured out the proof after they had figured out the answer, and everyone knows that the description of the experiment that's put in the paper is almost never how the experiment was actually done [2]:

A conversation with Cameron Neylon has got me thinking about how this relates to Software Carpentry. We don't know how scientists actually use computers day-to-day. We only know what they report after the fact, which (a) isn't much, and (b) is almost certainly a rational reconstruction rather than the messy reality.

What's more, traditional lectures don't show scientists how to navigate the false starts and circuitous by-ways of real coding. Instead, just as most mathematicians only show students finished, perfect proofs, most lectures and textbooks only show the program that worked, rather than its half-dozen incomplete or not-quite-right ancestors. (One of the reasons we now do most of our teaching via live coding is that it lets learners see the "two steps forward, one step back, google for a solution, and repeat" of real life.)

I think we need to teach people that there is a fundamental disconnect in all software development between repeatable, describable processes with flowcharts and checklists and the messy improvisation we all actually do. The problem is, many people crave the former even when they know it's a lie. As James Scott wrote in Seeing Like a State [3], large organizations almost always choose uniformity over productivity. From Stalin's forced collectivization of farming in the 1930s to the Rational Unified Process for software design, centralizers would rather have everyone do the same thing, even if it's wrong, than cope with the information overload and loss of control of individual agency.

The reason we don't teach people that rules are really just guidelines is that improvising requires judgment, which requires a deep knowledge of the problem domain and related solution techniques. Almost by definition, researchers are doing things that haven't been done before (at least, not exactly). This means that end-to-end "best practices" for scientific computing probably don't exist, because everyone's ends are different.

The best we can do, I think, is to show people the pieces out of which most of their competent peers assemble their daily work. What would help us do this is a collection of profiles of competent scientists' computational workflows—of how people who know what they're doing fit the pieces together when solving particular problems in particular domains. They would take time to produce (I think they'd have to be done by a neutral third party, since self-reporting is unreliable), and we'd have to be very careful that "this is how Jane does it" wasn't interpreted as "you must work this way", but showing people the actual process as well as its rational reconstruction would accelerate uptake of better practices like nothing else could.

1. Paywalled, of course, since that's how the IEEE helps ideas spread, but PDFs are available on the web.
2. This rational reconstruction is similar to program slicing, which is the process of extracting just those bits of a program needed to reproduce some specified subset of its output.
3. Scott's Two Cheers for Anarchism presents many of the same arguments in a shorter but more scattered way. If you only have time for one, read it instead.

Of the 29 people who responded to a brief questionnaire before a recent boot camp, we have:

They describe their current expertise as:

Here's how they thought they could do on some simple tasks:

1. Write a short program to read a file containing columns of numbers separated by commas, average the non-negative values in the second and fifth columns, and print the results.

   11	could do it easily
   15	could probably struggle through
   3	wouldn't know where to start

2. In a directory with 1000 text files, create a list of all files that contain the word "Drosophila", and redirect the output to a file called results.txt.

   12	could do it easily
   10	could struggle through
   7	wouldn't know where to start

3. Check out a working copy of a project from version control, add a file called paper.txt, and commit the change.

   3	could do it easily
   4	could struggle through
   22	wouldn't know where to start

4. A tab-delimited file has two columns: the date, and the highest temperature on that day. Produce a graph showing the average highest temperature for each month.

   19	could do it easily
   0	could struggle through
   10	wouldn't know where to start