Hour challenge: NCBI taxonomy tree

Today I'm planning a more science-y and useful hour challenge. Over the course of one hour, I'm going to transform the NCBI taxonomy from a tabulated text dump into a Newick tree which can be manipulated by phylogenetic tools. While other people have done this and Newick strings created from the NCBI taxonomy can be downloaded on external sites, the taxonomy is constantly updated, so it would be nice to have a reproducible process to update the tree whenever necessary.

I have other things going on and I haven't decided exactly when I'm going to do this today, but it'll happen. I think there's a good chance that this is the first one I actually finish in an hour, too. When I decide on the timing, and when I complete the project, I'll update this post with links. As always, everything will be done on GitHub so you can watch me tackle this live if you have nothing better to do.

While we're on the subject of the NCBI taxonomy...

Update 1: Busy day. The plan is to get started at 7 PM Eastern. So, theoretically, I should be finished by 8.

Update 2: Started at 7:15, and finished promptly at 8:15, so this was a success. This actually required fixing a bug in the BioPython Newick writer, as node labels in Newick trees weren't being quoted when they contained invalid characters such as spaces or parentheses. So, in addition to the 989,621-node NCBI Newick tree, I also generated a bug fix for BioPython.

The code is available at: https://github.com/bendmorris/ncbi_taxonomy

Hour challenge 12/7: zot, a command-line Zotero client

The last hour challenge was fun, but it was also a dismal failure - honestly, I had been envisioning nanote for a long time prior to developing it, and an hour was just not enough time to build in all the functionality I wanted. I've started using nanote in place of nano and have continued to build in additional functionality, and will probably continue to do so for a while. End result: I now understand how to write a program with the curses library, and my notes are much more organized than they were a week ago.

I use Zotero all the time to manage papers and books. Today I'll be developing a command line interface to Zotero. (There's not already one of these? Really?)

I'll be using the pygnotero library to interface with Zotero. To avoid the GPL, I'm not going to use pygnotero - instead, I'll just interface directly with Zotero's sqlite database. And, for fun, I'll use SQLAlchemy, which I've never used and should really learn more about. I want my client to be able to search (by author, title, citation, tags, etc.), add notes to papers, and output bibliographies (and potentially the text from PDF articles using pdfminer? I'm going to keep thinking about this.) It'll be called zot - short, memorable names for command line tools are always a good thing. I'm going to design it to pipe output to itself, i.e. "zot search ecoinformatics | zot bibliography" to generate a bibliography for all articles on ecoinformatics.

I'll begin sketching out planned functionality while I eat lunch at 12:00 (eastern time), start coding at 1:00 and hope to be finished no later than 2:00. Code will be available at https://github.com/hourchallenge/zot. The final result will be available on the Python Package Index as soon as it's relatively functional.

Update (2:00): my command line client can currently search for articles by title, author, or publication. I'm going to go for another hour to see if I can finish up.

Update (3:00): after two hours, I'm calling this finished. To try it out:

    pip install zot
    zot path /path/to/your/Zotero/directory/
    zot search --author Brown | zot bib

Hour challenge 11/30: nanote, a terminal note-taking app

I'm fairly stone age when it comes to productivity software - I keep track of notes, my calendar, etc. in plain text files on Dropbox and edit them with nano (yes, I use nano and sometimes gedit.) I've decided there's no excuse for this anymore, so I'm going to spend some time developing (open source) productivity tools that I can use from the terminal that are a bit more sophisticated.

Of course, I want to get something out of this, and I don't want to sink a lot of time into development. So, I'm starting a weekly "hour challenge." Every Friday afternoon, I'll be setting aside an hour to develop a specific terminal-based productivity app, and releasing the source on Github. I'll report on my progress here.

I was originally going to develop a terminal interface to Google calendar today, but I discovered gcalcli which fits my needs perfectly and there's no need to reinvent that wheel. So, this afternoon, I'll be developing a terminal-based note-taking system similar to Tomboy notes. I used to use Tomboy, but it frequently caused problems syncing with Dropbox and I constantly lost work, and it uses XML which makes trying to restore after conflicts prohibitive. My notes will use a simple markdown-based text format and allow easy linking to other notes and hierarchical note organization using directories. I'm going to borrow functionality from Nano and call my app "nanote." I'll be learning how to interact with the ncurses library in Python.

The rules: I'm allowed to look into what libraries to use, plan, etc. for one hour prior to starting to code. I won't actually start coding until exactly 1:00 PM, and at 2:00 it's pencils up. Obviously, I'll probably keep coding for a little bit after the fact, but at the end of the hour I want to have a bare bones, functional tool.

I've created an hourchallenge Github organization for storing these projects, so drop me a line if you want to participate! Suggestions for future tools are also welcome.

Shortly after 2:00, I'll update this post with the results and a link to the repository.

Update (2:00): I didn't make it, partially due to not enough planning/familiarity with ncurses and partially due to suddenly being offered free pizza. I'm giving myself another hour! I can definitely finish this off by 3:00.

The code so far is live at https://github.com/hourchallenge/nanote

Update (3:00): I got close! It's more or less functional: you can edit text, save, and link to other notes. I'll probably end up spending another couple hours filling out the missing functionality. Hey, I think this is pretty good for a two hour development sprint.

Update (12/1): after 24 hours, it does just about everything I wanted it to, and supports Markdown formatting. Mission accomplished!

So, in summary, I utterly failed to develop something in an hour (and maybe this project was too ambitious for an hour coding sprint) but I'm still pleased with what I was able to do. Tune in next week when I take another crack at it.

If you have an idea for a tool that you'd like to see developed, or want to develop a tool in an hour yourself, let me know!

nanote is now available on the Python Package Index, so "pip install nanote && nanote" should get it running.