Here’s a typical problem from the book. Put a block of ice in a bathtub, then fill the bathtub to the brim with water, so that the block is floating freely. When the ice melts, will the water level go up (causing a spill), go down, or stay the same? Hm… well, the ice displaces its own weight of water, so when it melts, it exactly fills the “hole” it made, so the water level stays the same.

Now let’s try something a bit more complicated. Put the same block of ice in the bathtub, but put an iron weight on top of it, and then fill the tub to the brim. Now what happens when the ice melts? Does the water level go up, go down, or stay the same? Epstein would say that if you can answer questions like that, then you can think physically. It isn’t about calculation—as with Dan Meyer’s “Joulies“, it’s about understanding principles and following them through to conclusions.

The real aim of Software Carpentry is to teach scientists to think like that about computing. We want people to understand the principles:

• model-view separation
• human-readable vs. machine-readable data
• copying vs. aliasing
• state machines
• different models of computation (imperative, functional, reactive, declarative)
• interface vs. implementation
• the complementarity of algorithms and data structures
• code as data (and data as code)

The Unix shell, Python, SQL, regular expressions, and what not are how we hook people (“Hey look, something useful”) and how we get these principles across (as with most big ideas, any direct description is either incomprehensible or banal). However, these principles aren’t natural laws in the way as F=ma and the Second Law of Thermodynamics—if you compare them to the principles I listed a month ago, there’s overlap, but the lists aren’t the same. So:

1. What is the best way (or a good, stable way) to carve up this intellectual space?
2. How do we tell what a particular person actually understands?

“Write this program” is not an answer to the second problem—as many studies have shown, people can solve routine problems by rote without really understanding what they’re doing. (This is the starting point for Eric Mazur’s work on peer instruction, and the reason so many of us are so skeptical about things like the Khan Academy.)

Can we just teach the tools (for some value of “just”) and let the big ideas sort themselves out? The answer is clearly “yes”, because that’s what I did from 1998 to 2007. Does it work? I think the answer is “only partially”:  some people generalize from specifics to principles correctly on their own, but many either don’t do it at all, do it incompletely, or do it incorrectly. And does it matter? I think so: I think that if we want scientists (or anyone else) to use computing on their own, for their own ends, they need to be able to step past what we’ve shown them with good odds of success, and that certainly requires understanding “why” as well as “what”.

For example, consider this question about Subversion:

Emmy wants to see what has changed in her working copy since revision 120. The command she should run is:

1. svn log -r 120
2. svn diff -r 120
3. svn revert -r 120
4. None of the above

It addresses Q05 (“How can I keep track of what I’ve done?”) fairly directly, but not R02 (“Use a version control system”), R03 (“Automate repetitive tasks”), or any of the basic principles. Open-ended answers might do the latter, but it’s hard to come up with ones that don’t lead the witness: asking, “When would you use a version control system?” isn’t going to give us much insight into what people actually think. We could combine a few multiple-choice with a few open-ended, but realistically, if it takes more than 10-15 minutes for people to answer, many (most?) won’t. If anyone can see a way to square this circle, I’d welcome ideas.

	for people who have mastered our core content
	for people who have organized and run workshops
	for people who have created content

Please let us know what you think.

When we started Software Carpentry back in the late 1990s, we used Perl as a teaching language instead of Python. At the time, it was a no-brainer: Perl had many more users, better documentation, and more libraries. We switched because we found ourselves explaining the same inconsistencies over and over again (as I’ve said many times since, every page of the O’Reilly Pocket Guide to Perl used one of the words “except”, “unless”, or “however” at least once). Python had fewer “buts”: we saw right away that students were learning concepts more quickly, and they seemed to retain more as well.

But Python isn’t perfect, and I was reminded very forcefully of its biggest flaw on Saturday, when I spent half a day teaching kids aged 8-14 how to program as part of a Mozilla Hack Jam in Toronto. About three quarters of the kids were able to start drawing pictures with Python’s turtle graphics library right away. The other quarter, though, stumbled (and were sometimes blocked completely) by the same old installation headaches that plague grownups trying to use Python to do science.

One would-be learner showed up with a brand-new MacBook Air running OS X 10.7. Half an hour and four downloads later, he still couldn’t get a turtle to draw a straight line.  We tried 32 and 64-bit DMGs for Python 2.7.2 and Python 3.2.2, without luck; the only advice Google found for us started, “Install the latest version of XCode…”, at which point we gave up. Several others, who had Windows 7 machines, were able to install, but then we discovered that Python still doesn’t put itself on the search PATH. “Oh,” said one of my helpers, “That’s easy, you just go into System… then Advanced… then edit this environment variable…” It’s a good thing he was looking at the computer as he said this, instead of at the faces of the kids he was trying to help—if he’d been doing the latter, he would have realized how inappropriate “simple” and “just” were.

People used to talk about “grand challenges” in scientific computing. Mostly, they meant the kind of big science that shows up on magazine covers. For me, though, the only “grand challenge” in scientific computing that matters is making stuff work the first time for everyone. It might not be as sexy as protein folding, global climate change, or predictive models of fender crumpling, but it would help a lot more people—and not just scientists.

Later: another stumbling block when doing things with turtle graphics is Python’s “counted loop” idiom:

for i in range(3):
  do something

If I want people to draw squares, hexagons, and what-not, I either wave my hands (“Trust me, this is just what you do”) or explain functions and lists when what I really want to do is explain loops. It’s not as big a thing as the installation headaches, but first-class ranges:

for i in [0:3]:
  do something

would make things noticeably easier in this one particular case. Is it important enough to merit changing the language? Probably not on its own, but if there are other reasons to do it—or to go all out and add a cross-product operator:

for (i, j) in [0:3] @ [0:5]:
  do something 15

As a bonus, we could then overload @ for matrix multiplication

Today, we are starting Python. You can follow our material on our github page’s wiki. Pay particular attention to Katy’s lectures on git and github, where she introduced git for personal use in hour one then introduced collaborative use through github in the second hour.

Here’s a picture (more to come!)

Titus Brown: Choosing Infrastructure and Testing Tools for Scientific Software Projects

Cameron Neylon: A Web Native Research Record: Applying the Best of the Web to the Lab Notebook

Michael Nielsen: Doing Science in the Open: How Online Tools are Changing Scientific Discovery

David Rich: Using ‘Desktop’ Languages for Big Problems

Victoria Stodden: How Computational Science is Changing the Scientific Method

Jon Udell: Collaborative Curation of Public Events

Greg Wilson: Opening Remarks

In addition, while the scientists and engineers we’re trying to help might think that computing is interesting, their real passion is quantum chemistry, neurology, or climate change; in practical terms, computing is a tax they have to pay in order to do the research they actually want to do [2].  From that perspective, “wander around and stumble upon” feels like a high-risk strategy, so they (mostly) vote with their feet and don’t do it.

Second, even those who do wander and stumble tend to find very different things. As a result, there’s no common core of skills or assumptions that one researcher can reasonably expect her peers to be familiar with. In contrast, most researchers can expect colleagues to know at least a few basic things about statistics, and to share some cultural values about when a correlation is significant and so on.  In choosing what to include in our core, we’re also (implicitly) making a statement about what that core is, and what’s reasonable to expect others to share.

[1] What’s really interesting, though, is the discussion in the first few minutes about Silicon Valley’s ed-tech amnesia.

[2] Regarding Hargadon’s comment about “willingness to hack”, I think that every researcher I’ve ever met has that in spades—they’re just investing that energy in something other than programming. And yes, lists of “things programmers need to know” make me yawn too—but only if I already know enough about the topic to forge ahead on my own. I’m really grateful for “must read” lists whenever I dive into a new area…

If you’d like to help, please:

1. Volunteer to be recorded by mailing us. We’ll help you install a screen recorder (if you don’t have one already—you might be surprised to find that you do), give you a small problem, and edit the video you produce so that you don’t have to.
2. Volunteer to edit video for us, so that we can put our energy into organizing people .
3. Volunteer to work the floor at PyCon in March. We can’t attend (workshops to run, etc.), but it would be great if we could get a dozen or more “here’s how I do it” recordings done during the conference.

Remember, as an open source project, Software Carpentry depends on your help to survive and thrive. If you have wanted to help, but have worried that creating and recording lectures would be too much work, this is a way for you to help that will take half an hour or less. We look forward to hearing from you.

http://bioinformatics.msu.edu/ngs-summer-course-2012

June 4th – June 15th, 2012
Kellogg Biological Station, MSU
Course sponsor: NIH.

Instructors: Dr. C. Titus Brown, Dr. Ian Dworkin, and Dr. Istvan Albert.

Board of advisors: Dr. Kevin White; Dr. Paul Sternberg; Dr. Rich Lenski; Dr. Robin Buell; Dr. Jim Tiedje; Dr. Lincoln Stein

Applications are being accepted through March 1st (midnight Pacific)!

Course Description

This intensive two week summer course will introduce attendees with a strong biology background to the practice of analyzing short-read sequencing data from Roche 454, Illumina GA2, ABI SOLiD, Pacific Biosciences, and other next-gen platforms. The first week will introduce students to computational thinking and large-scale data analysis on UNIX platforms. The second week will focus on mapping, assembly, and analysis of short-read data for resequencing, ChIP-seq, and RNAseq.

No prior programming experience is required, although familiarity with some programming concepts is helpful, and bravery in the face of the unknown is necessary. 2 years or more of graduate school in a biological science is strongly suggested. Faculty, postdocs, and research staff are more than welcome!

Students will gain practical experience in:

• Python and bash shell scripting
• cloud computing/Amazon EC2
• basic software installation on UNIX
• installing and running maq, bowtie, and velvet
• querying mappings and evaluating assemblies

Materials from last year’s course are available at http://ged.msu.edu/angus/tutorials-2011 under a Creative Commons/use+reuse license.

You can read a blog post about last year’s course at http://ivory.idyll.org/blog/jun-11/ngs-2011.