Ted Hart

I’m a senior data scientist in silicon valley and adjunct faculty at the University of Vermont. I build things for data: things that process it, parse it, visualize it, and analyze it. I like my beer cold, my snow deep, my mountains high, and my data open. I am a recovering academic.

Ted Hart


ecologist / data scientist / developer

A river of tail lights

Traffic pours on to I-280 as we head home. It always grinds to a halt at the same places, as people rush north, heading home from Stanford and Facebook. A curious feature of the Peninsula (that sliver of land west of the bay that is bracketed by Daly City to the north and San Jose to the south) is that company name is synonymous with town. Mountain View is Google, Palo Alto is Stanford (true not a company like the others), Menlo Park is Facebook, Los Gatos is Netflix and the list goes on. My distaste for a given company is now proportional to how much traffic it causes (for me anyway. So I curse Stanford as we drive towards the Palo Alto entrance ramp and am greeted with a river of glowing tail lights. So many people squeezed on to such a small spit of land, but I can't begrudge them too much. After all, it's a modern day boom, and I've come rushing here like everyone else.

But why have I come rushing here? (In my last post, I had left Vancouver and my post-doc to work at NEON). That remains the hardest question for me to answer about this whole journey. To be fair I had something of an ideal position at NEON. I was able to go to conferences, develop software, do exciting informatics work, my back wasn't against the wall with funding, I lived in a great location, and I worked with awesome people. So why did I leave? An entire bottle of Knob Creek went into that decision, and I'm still not sure I have clarity of thought (maybe another bottle of whiskey would help?). Here are some of the things that I did think about:


I weighed my options, stay at NEON or leave. My wife asked me which decision was most reversible if I was really unhappy. I realized that I could probably find a place in the world of scientific software again, but where I work now would only knock once, and if you didn't answer then they wouldn't knock again. Does the scientific community look down on industry experience? I'm not sure, but since I no longer harbor ideas of a tenure track job, it seems like the kind of role I'd be interested in academic science would be open to my varied career path (However that remains to be seen).

Distributed community

Much of what I really like about science is distributed now; it's on twitter, it's on github. I could still contribute to projects I believed in like Data Carpentry, Software Carpentry, rOpenSci. Actually things did not go well with that last one (it was a bitter pill to swallow, and not one I wanted to). The take home message? Some doors will stay open, others will close. Another step I took was to get an adjunct appointment at my old university (Univ. of Vermont) so I can still publish now and then, and still work on some software projects on the side. So even though I work in private industry, I still feel connected to academia.

Career mobility

If I were a betting man, I'd say it's only a matter of time before the typical data scientist isn't someone with a PhD in Physics (or biology), but someone who got a 1 year masters certificate. There's just too much press about data science and too much money to be made by places like Berkeley (which has an online data scientist program) minting data scientists with quick degrees for this not to happen. I figured if I was going to move to industry, the iron was hot so to speak. In three years I think the hiring landscape will be very different. But if I get in now at a good company I'll have a good chance to advance there before the glut, or be senior enough to move to a different company.

I've spoken to many people now about this transition, many people e-mailing me out of the blue for skype chats. They ask a lot about the skills they might need, where to find job postings, it's all very practical. Often missing from the conversation is that no matter what, you're walking away from a world you've likely known from 6-8 years. All that social capital you've amassed through collaboration and conferences? Gone. You are walking into a world that might not know what a post-doc even is, and certainly does not give a shit about where you publish. So what weight do you give to the personal cost of the transition? What weight do you do you give to the practical? In the end even after that bottle of Knob Creek, I'm not sure I have an answer.

And that's where I'll close the book on this. If you read part 1 and part 2, you'll see they're quite personal. I had thought initially that I would write a series of posts about how the academic system in broken. Instead I found that my reasons were much more personal than structural. But I'd encourage anyone contemplating jumping off the academic ship to consider the costs and benefits beyond just the utilitarian.