The C++ trap

I came across this wonderful piece of historical retelling by David Beazley, one of my favorite Pythonistas and the author of Python Essential Reference. Here is a man who conquered C++ in just about every way, but ultimately found himself trapped in its byzantine complexity, only to escape by way of Python.

Swig grew a fully compatible C++ preprocessor that fully supported macros. A complete C++ type system was implemented including support for namespaces, templates, and even such things as template partial specialization. Swig evolved into a multi-pass compiler that was doing all sorts of global analysis of the interface. Just to give you an idea, Swig would do things such as automatically detect/wrap C++ smart pointers.It could wrap overloaded C++ methods/function. Also, if you had a C++ class with virtual methods, it would only make one Python wrapper function and then reuse across all wrapped subclasses.

Under the covers of all of this, the implementation basically evolved into a sophisticated macro preprocessor coupled with a pattern matching engine built on top of the C++ type system […] This whole pattern matching approach had a huge power if you knew what you were doing […]

In hindsight however, I think the complexity of Swig has exceeded anyone’s ability to fully understand it (including my own). For example, to even make sense of what’s happening, you have to have a pretty solid grasp of the C/C++ type system (easier said than done). Couple that with all sorts of crazy pattern matching, low-level code fragments, and a ton of macro definitions, your head will literally explode if you try to figure out what’s happening. So far as I know, recent versions of Swig have even combined all of this type-pattern matching with regular expressions. I can’t even fathom it.

Sadly, my involvement was Swig was an unfortunate casualty of my academic career biting the dust. By 2005, I was so burned out of working on it and so sick of what I was doing, I quite literally put all of my computer stuff aside to go play in a band for a few years. After a few years, I came back to programming (obviously), but not to keep working on the same stuff. In particularly, I will die quite happy if I never have to look at another line of C++ code ever again. No, I would much rather fling my toddlers, ride my bike, play piano, or do just about anything than ever do that again.

From this Swig mailing list entry.

import this: learning the Zen of Python with code and slides

It’s hard to find me gushing more unapologetically than when I talk about the virtues of my favorite programming language, Python.

Indeed, my life for the last 3 years has been dominated by the language. In many ways, pursuing a startup and enduring the associated financial hardship was partially because I had become frustrated with using Java in my full-time work and wanted to convert hobby projects I was building outside of work hours into full-fledged projects.

Continue reading import this: learning the Zen of Python with code and slides

Engineers don’t become engineers

And, sadly, our top engineering graduates don’t always become engineers. They move into finance or management consulting — both of which pay far higher salaries than engineering. I have seen the dilemma that my engineering students at at Duke University have faced. Do they take a job in civil engineering that pays $70,000, or join big Wall Street financial firm and make $120,000? With the hefty student loans that hang over their heads, most have made the financially sensible decision. In some years, half of our graduates have ended up taking jobs outside of engineering. Instead of developing new types of medical devices, renewable energy sources and ways to sustain the environment, my most brilliant students are designing new ways to help our investment banks engineer the financial system.

[…] We also need to make the engineering profession “cool” again, with the same sense of excitement and urgency in engineering and science that we saw during the Sputnik days. Back then, engineering was considered essential to the nation’s survival. Engineers and scientists were national heroes. It’s not that we don’t have problems to solve. The economy is in dire straits. Natural resources such as food, water, and crude oil are becoming scarce. Drug-resistant bacteria threaten us with doomsday plagues. But we’re not offering our best minds incentive to solve them.

From Mr. President, there is no engineer shortage.

Luckily this is happening already in high tech in NYC, thanks to awesome programs like HackNY and collabraCode (both of which my startup Parse.ly formally supported). As much as it pains me to say it, I also think The Social Network may be seen as a cultural catalyst for software engineers becoming “cool”.

But high tech is only a small piece of the puzzle — we need the same active marketing for students’ minds in biotech, education, medical research, civil engineering, etc.

Turning 27

Today, I turn 27. Even though I was deep in the middle of a project late last night, I peeled myself away from my monitors, went to sleep, and woke up late to enjoy a day of reading outside.

Parse.ly has an official “take your birthday off” policy, so I made sure to set a good example.

I remember when I was younger, I used to look forward to birthdays very eagerly. Birthdays were when I got a new videogame or programming book. Birthdays were about stuff, and taking the day to play with new toys.

Now, over a decade later, my birthday is much less about stuff. I don’t play videogames anymore, and I already know how to program. I am fortunate to live comfortably and don’t long for stuff any longer. My Nintendo Wii gathers dust (like everyone else’s, it seems). My computer is no longer used to amuse me, but to allow me to work on my passions — building software, building a company, staying informed, informing others. I have a seemingly endless queue of books I’d like to read, movies I’d like to watch, things I’d like to write, software I’d like to build. I’ve come to realize that birthdays, at my age, are more about time.

In my ruthlessly efficient worldview — where I regularly talk of cost-benefit analysis, backlog prioritization, and productivity — my birthday has become about taking a moment to flip my prioritized world on its head. Let’s not pick an item from the top of the prioritized backlog. Instead, let me take something from the backburner, for once. Let me behave — if only for a day — as if I had all the time in the world.

I don’t need stuff. I just need time. Of course, that’s the bittersweet part of one’s birthday. That even as you come to realize the importance of time, the day acts as a reminder of how our time on this earth is limited. 1 day passes, and only n-1 left to make a difference.

Upcoming: standing desk setup, Python training, Groovy/JavaScript articles

I’ve been quite busy with work lately, so haven’t had time to send a few posts toward my blog. However, I have been working on some spare time and work-related projects that I’d love to share with everyone here.

Among them:

  • Lifehacking through standing desks. I have created a standing desk setup for my home office, and my investors in Parse.ly have created a healthy startup office in NYC. I have some thoughts about this as it applies engineering and hacking to the thing many of us do most: sit in a desk chair all day.
  • 2-day Python Training Course. I have created a 2-day Python training course that helps existing programmers learn Python by comparing it very directly to languages they may already know, like C and Java. I gave this course to a group of government employees a few months ago. It has some interesting characteristics: I designed the whole course using ReStructured Text (ReST) and compiled it into a web-based presentation. This means that the entire course has the potential to be “open source” — exercises, slides, and all. I plan to release this to the public. For now, I am just clearing a few of the images I used in the course to make sure I don’t inadvertently infringe copyright. After that, I will open to the public.
  • Groovy/JavaScript articles go public domain. Some articles I wrote for GroovyMag and JSMag last year are now able to be published in the public domain. These include one about RESTful services with Groovy, one that talks about functional programming with JavaScript, and finally one that discusses a design for “metagrids” in ExtJS 3.x. I will put all three articles up on this blog once they are reformatted.

Stay tuned!

Capital, public good, and the real economy

Every once in awhile, I get this strange feeling that the entire economy is a sham, my optimism in capitalism and entrepreneurship a farce, my own path much more manufactured by a system than carved out on my own. Luckily, these moments are fleeting, so I am able to go on with my life, pursuing my goals confident that the end result will be substantial public good, and even a personal gain.

One of these moments happened recently when I watched the documentary An Inside Job. Though the facts presented by this excellently-shot film were not new to me — I had heard the same facts in the writing of Matt Taibbi, the Giant Pool of Money special on NPR, the terribly underrated film The New American Ruling Class, and on blogs such as Naked Capitalism and 13 Bankers — no other single source of information has so connected the dots between the political, academic, private and public problems which culminated in the financial crisis we witnessed in 2008.

Joseph Stiglitz recently wrote an article about the richest 1% of Americans, and their increasingly better relative wealth and income over the last 30 years. His points are well-argued (and again, nothing new for those well-versed in history of economics of neoliberalism). But, perhaps most interesting is Stiglitz’s recognition of the shrine of finance.

To give just one example, far too many of our most talented young people, seeing the astronomical rewards, have gone into finance rather than into fields that would lead to a more productive and healthy economy.

John Maynard Keynes also had something to say on the subject:

For my part I think that capitalism, wisely managed, can probably be made more efficient for attaining economic ends than any alternative system yet in sight. But, capitalism in itself is in many ways extremely objectionable. [… As for] Economics, it is a very dangerous science.

When I swirl that together with my vantage point in 2011, looking back on the crisis of 2008, I find it very hard to not wish we were doing a rethink of the system altogether.

It is not enough that the conditions of labour are concentrated in a mass, in the shape of capital, at the one pole of society, while at the other are grouped masses of men, who have nothing to sell but their labour-power. Neither is it enough that they are compelled to sell it voluntarily. The advance of capitalist production develops a working class, which by education, tradition, habit, looks upon the conditions of that mode of production as self-evident laws of Nature.

The organisation of the capitalist process of production, once fully developed, breaks down all resistance. The constant generation of a relative surplus-population keeps the law of supply and demand of labour, and therefore keeps wages in a rut that corresponds with the wants of capital. The dull compulsion of economic relations completes the subjection of the labourer to the capitalist.

Direct force, outside economic conditions, is, of course, still used [at times], but only exceptionally. In the ordinary run of things, the labourer can be left to the “natural laws of production,” i.e., to his dependence on capital, a dependence springing from, and guaranteed in perpetuity by, the conditions of production themselves.

But, then I snap out of it. Of course, I don’t have to tell you who wrote that quote and what that thinking led toward in history.

Groovy, the Python of Java

I was a bona fide Java programmer for 5 years before I started working on Aleph Point and Parse.ly. I truly believe that Python and JavaScript are fundamentally better languages than Java for a variety of reasons born out of experience with each of them. (Note: Before this gets marked as flamebait, please notice that not only was I Java programmer for more than 5 years, but I was also a Java open source contributor!) I have enormous respect for the Java open source community, which has produced some of the highest quality modules available anywhere.

Now, don’t get me wrong — Python also has batteries included, and usually, when I think that I’m missing a great module I used to use in Java, it already exists in a much more powerful form in Python’s Standard Library or the wealth of modules on PyPI, GitHub, and Bitbucket. However, I believe in not reinventing the wheel, and so if a great open source tool exists in Java, I will want to interact with it.

One of these modules which we use extensively at Parse.ly is Apache Solr, and its surrounding Lucene project modules. Lucene is an extremely mature framework for document indexing, and Solr is a powerful server-ization of that technology that fits well into complex, mixed language distributed systems. I know there are efforts — like Whoosh — to build fast search engines atop the Python language. And I applaud these efforts — more projects means more competition, and more competition means better products. However, I still believe that you go with the best of breed tools available for production software, and you try not to let religious arguments about programming language get in the way.

Lately, I have come across more and more Java open source projects that have no equivalent in Python, and which I would like to access. Knowing that I wanted to feel comfortable incorporating Java open source projects — beyond Solr, which was already nicely wrapped as a web service — I, at first, thought that I’d be forced to still live among the weeds of complex class and interface definitions, cumbersome Java IDEs, XML configuration files, and (IMO) time-wasting rabbit holes like dependency injection, configuration management, and classpath hell. And then I found Groovy.

Continue reading Groovy, the Python of Java

Startups: Not for the faint of heart

Early on during this startup adventure, a person I trust told me, “Watch out — startups aren’t for the faint of heart.” Looking back on my personal net income graph from 2009 to present, I can see what he meant.

May 2009 is when I entered Dreamit Ventures to begin working on what would become Parse.ly. That’s when I plunged my “savings buffer” into the company. The few months after that had me frantically trying to recover from the realization that startup progress is measured in months and years, not days and weeks.

Sachin and I switched gears from targeting consumers with a free product to targeting large online content properties with a paid product, and bootstrapped the company with side consulting gigs. We didn’t tell anyone we did the side consulting work (unless they specifically asked). We watched other entrepreneurs go into credit card debt and borrow money from trusting friends and relatives. We didn’t believe in that, so we took the hard road of “earning our survival”.

However, our costs were going up, not down, as we pursued a more ambitious product with more demanding clients. Also, my expenses skyrocketed as COBRA disappeared for my health insurance and I had to pay for horribly overpriced sole proprietorship plans. (Fact: America’s broken healthcare system is harmful to entrepreneurs.) I knew I needed to do something to “stop the bleeding” on my financial situation — so, I took on more consulting gigs…

Continue reading Startups: Not for the faint of heart

Understanding Wisconsin protests with big language data

I made an interesting discovery today.

“Free Market” vs. “Labor Union” in Google Ngram Book Viewer

Explains why no one has heard of labor unions and everyone is raving about the free market 🙂

(by the way, you can download the entire dataset behind this neat little Google Labs project)

What’s the best part about being a software engineer?

I wrote this in response to the question, “What’s the best part about being a software engineer?” It was originally posted to Quora, but I figured Quora might disappear from the web someday. So here is my response, for posterity:

Being a software engineer gives you access to the only career choice where you can simultaneously rid the world of a problem, create a scalable business, satisfy a personal creative hunger, and have an effect on millions of people’s lives. That is the career choice of founding a software product company.

And today, you can do this with comparatively little personal capital investment than times in the past.

There’s nothing like it. I wrote about my path to this realization in my essay, “What One Does”.

I also believe software engineering, per se, (that is, coding) is an extremely fun activity. Software design is complex, multi-disciplinary, and demanding. The web and open source have created an incredible sense of community and shared wisdom around this activity. In my view, we’re lucky to be able to be compensated in any way for doing it. I’d do it for free.

Update from the future in 2025: I still feel the same way.