Solving “accidents” and “essences” of programming with better languages

The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures. Yet the program construct, unlike the poet’s words, is real in the sense that it moves and works, producing visible outputs separate from the construct itself. It prints results, draws pictures, produces sounds, moves arms. The magic of myth and legend has come true in our time. One types the correct incantation on a keyboard, and a display screen comes to life, showing things that never were nor could be. … The computer resembles the magic of legend in this respect, too. If one character, one pause, of the incantation is not strictly in proper form, the magic doesn’t work. Human beings are not accustomed to being perfect, an few areas of human activity demand it. Adjusting to the requirement for perfection is, I think, the most difficult part of learning to program.

Frederick Brooks, author of the The Mythical Man-Month, disinguished between two problems in software development: the accidental problems and the essential ones. (Read Brooks’ watershed article, “The Silver Bullet“.) The industry likes to argue that we have made huge leaps in terms of the accidental in recent years. Programmer productivity is better with the help of better software revision systems, but more importantly there are RAD tools like GUI designers, powerful IDEs like Eclipse/VS.NET. Do I think we’re headed in the right direction?

In a sense. I think those tools are useful, but only because I think all we can do is shrink the accidental problem as small as possible, and then do our best to tackle the essential. The major win in terms of the accidental and even the essential is high-level programming, and I mean really high level, like Python. Why is this important? Well, Brooks once talked about how PL/I was a great software engineering language because the statements written to the compiler are “pretty close” to the “thought stuff” the programmer is working with in his head. But PL/I is nowhere near there (neither is C++, or even Java or C#). However, Python heads in that direction.

In my head, unless I really am immersed in C code all the time, I don’t think of things like types, memory management, or, God, pointer arithmetic. When we solve problems, we solve it in something computer scientists have used for years: pseudocode. Pseudocode is nothing more than code that expresses an algorithm or approach without worrying about the gory details of the underlying hardware.

Python isn’t quite pseudocode, but it’s closer than anything else I’ve seen. And that’s a huge help to productivity not just from the point of view of the single developer, but, I’d argue, also from the point of view of a team of developers. Code readability is hugely important in team environments. C and Perl can be very unreadable, which makes them ill-suited to team development. Java and C++ are better, but even they suffer from some readability problems (where the actual solutions can be masked by “best practices” and strange ways of exploiting constructs of the language). Python can be hacked too, but it tends not to be, and it tends to have very high readability.

But what about the tools mentioned above (Eclipse/VS.NET)? I often wondered about all the *nix programmers (including many of my Computer Science professors) who still load up X11 and dump into vim or emacs for their programming needs. Wouldn’t they benefit from the latest and greatest in IntelliSense, Refactoring, CallTips, SaneOnlineDocumentation, and any other CamelCase ideas I can think of for a development environment? Most *nix users would say that stuff is unnecessary–and in many senses, they are right. The essential problem remains, no matter how fancy your IDE is.

A lot of *nix developers shrink their accidental problem on a “as-needed” basis, by coding plugins or complex configurations for their highly-extensible text editors. But nowadays, productivity is starting to be a concern even in the OSS world, where things have traditionally stuck with the old “mortar and pestle,” or, should I say, “gcc and gdb”. That’s why we see projects like Mono gaining a lot of popularity.

But Python offers a nice, alternative path, I think. The accidental problem is worth shrinking, and I think it can be done simply by one major refocus:

(1) make the language usable, not the tool. (IMO, already done with Python)

and

(2) create source code analyzing tools that integrate with development environments to make the language-bearer more productive.

You can see enormous success in (2) with the Eclipse IDE project. My main problem: Eclipse’s focus is on a language that isn’t very usable. Java is certainly better than C, but still, it has significant problems that stop it from connecting me (the programmer) with my problem-solving ideas (the thought-stuff of programming).

At my new job, I work with Java eight hours a day, but my code just doesn’t read and work like my Python code, which is much closer to the underlying algorithms and approaches. Java certainly pushed forward OO on the masses of computer programmers, but it did so without learning non-OO but very handy scripting language concepts, like having Lists and Hashes be built-in types, with tons of syntactic sugar, for crying out loud. Most problems break down to List and Hash problems, after all. What’s more, OO on its own isn’t a panacea: I’ve seen many convoluted OO designs that I wish were written procedurally, and which might have been more performant if they had been. The software engineering community thinks that there will be a silver bullet (OOP! AOP! XP! MDE!), but I think just getting programmers closer to the “thought-stuff” will be an effort with better rewards.

The Unkindest Cut

A relatively unsophisticated article on circumcision showed up on Salon a few days ago. It’s entitled “The Unkindest Cut” and is about the conflict of a Jewish father between his Jewish mother and his non-Jewish wife over the issue of whether to circumcise his newborn son. Although it mentions some of the history of circumcision in the United States, it doesn’t go into nearly enough depth about how strange and barbaric the practice is. A letter that came into Salon from a reader has some good points, however:

This is one of the hottest parenting issues (along with breastfeeding and sleeping). I am not surprised that Salon is already flooded with letters and the emotions are rising high on both sides. I am the mother of a 3-year-old boy. He is uncircumcised and the idea of having him circumcised never even crossed my mind. I am from Europe, therefore circumcision is not part of my culture. My husband is a Hindu from India, so it isn’t part of his culture either. There is more and more evidence that shows that circumcision is an unnecessary procedure. There are more and more organizations and individuals trying to educate the public about this. Two of my favorites are: www.jewsagainstcircumcision.org and www.nocirc.org.

There is lots of very useful information regarding this topic. There are a few points I’d like to make for the sake of argument against it. Some fathers say: I want my son to look like me. So if you had a finger, a hand, or an arm missing from birth or as a result of an accident would you want to chop off your child’s corresponding body part just to make him look like you? (Sorry this is not my own idea but I like it a lot). The other point is my original thought: we, as Western society are outraged by the practice of female circumcision (mostly practiced in Africa and some predominantly Muslim areas elsewhere). What is the difference? That female circumcision is not a tradition in our culture. So it’s O.K. to keep mutilating our boys as long as we leave our girls alone… How hypocritical! And on top of that both traditions originate on the same basis: to reduce sexual pleasure and the desire to masturbate and enjoy sex.

As per some first person accounts from men who grew into adulthood intact and then got circumcised, they tell exactly how much less pleasurable sex is afterwards… Do a search on your favorite search engine for more info on the topic.

So as a parent of a boy I will leave my son’ penis alone and will make sure that everybody else does until he is old enough to make a decision about having his own body part cut off (which I think would only occur if he ends up having problems with having foreskin and sex would be unpleasant or painful).

I am not condemning people who think differently. I simply feel sorry for their baby boys… I hope one day we’ll come to our senses about this painful and inhumane practice.

As for the author, I feel sorry for him too, that he had to go through this emotionally painful experience to come to understand that he made the wrong choice.

I have a lot more to say on this topic (in fact, in college I wrote long research papers on the topic, including some original research into grotesque Victorian age masturbation control techniques, which were the precursors to routine circumcision), but probably won’t get the time to write it up. If you’re wondering about it, drop me a line.

Conversation on Torture with Doomsy

Been having a cross-blog discussion with Doomsy over at the Liberal Doomsayer about our recent crimes torturing detainees, as widely reported in NYTimes and on Alternet. A follow-up to this post from me that I figured I should post again here:

The thing is, I don’t think we ever can redeem ourselves. America hasn’t been a saint throughout its military and political history, but as many leftist columnists are now pointing out, we have dropped our bar so far below the one we set at Nuremberg that it’s hard to see how the world can forgive us, never mind we, the people, forgiving the US Government. Saddam, a dictator we helped create and helped carry out his war crimes, was executed in an instant, in the most inhumane way. A member of the military I recently interviewed told me [paraphrasing] that “every military officer knew full well that Saddam would be executed the second he was turned over to the ‘Iraqi Government’,” and those quotes are his, not mine. In his mind, and he has been in West Baghdad for the last year fighting on the front lines, the “Iraqi Government” is nothing more than a a few corrupt politicians and a few importantly-placed American agents. “We’ve turned over detainees who weren’t even proven guilty of their crimes in Iraq, and the ‘Iraqi Government’ murdered them with a shot in the head before we were even out the door. We’ve all come to understand that ‘handing someone over to the Iraqis’ is doublespeak for ‘send that person to die’. Who physically pulls the trigger is really an irrelevant detail.” So I don’t want US Government officials telling us this is “their [the Iraqi’s] system, their method of justice.” It’s ours, the blood is all over our hands. The fact that we torture should come as no surprise. And the case of Donald Vance (note: an American contractor who blew the whistle on his employer in Baghdad and was held and tortured by our military) just shows that no one is safe, that we don’t reserve our techniques for those we consider “evil”, but that it has just become a routine process for our military operations.

“The Good Shepherd”: A Good Film Concept, but Lacks a Pulse

I have to agree with Peter Travers’ review of The Good Shephard. The movie is not a complete failure: it’s beautifully made, and beautifully acted. Unfortunately, it is just not well-written.

We go through three hours of meeting character after character, in different time periods, sometimes the same character appears twice with different names (due to double-agentry), and at the end of it all you feel that the only character who got a bit of development was the leading one, Matt Damon’s, who hardly spoke a word throughout the movie.

I tried hard to like this movie, I really did. But I think it’s a crime to employ this much acting and cinematography talent, and end up with a movie that says little else about life inside the CIA but the spy cliche, “trust no one.” The movie is full of visually memorable scenes, but absolutely no memorable dialogue.

The movie should have been more focused, had less characters, and covered less ground. DeNiro: either you make a seven hour epic, or you make a Hollywood two hour film. You can’t have your cake and eat it too, I’m afraid.

Free and DRM-Free E-Books

I got a Nokia 770 for Christmas, to replace my aging and dying Palm V. I’ll probably write more about the 770 and the Palm V later, but suffice it to say that I got the Palm V when it came out, approximately 10 years ago, and have used it ever since. My main uses for it were AvantGo and Vindigo, although in the past I did get other uses out of it as well. I plan to keep the Palm around mainly for the Vindigo features, which I haven’t seen done better anywhere else. (No book on the planet has as much information about NYC as my 8MB Palm V with Vindigo loaded on it.)

I basically got the 770 because of all the great stuff I heard on Teleread and other sites about its use as a portable e-book reader. Indeed, that has been the most pleasant experience on the device. With the Evince PDF viewer and with FBReader (an Open Source ebook reader for a lot of different formats, including HTML, Plucker, and zTxt), I am able to read a ton of stuff while I’m in my commute, and all on a beautiful 800×480 screen that fits in the palm of my hand. (Yes, finally I get to read with serif, anti-aliased fonts!)

I have been overwhelmed with how many free e-books you can find online, thanks in part to efforts by groups like Creative Commons. Here are some good ones:

Feel free to post more in comments!

Embarassed to be an American

Following up on the torture table from my last post, today I read two of the most engrossing and depressing articles I’ve ever read, about detainees tortured by the US Government, using cruel and unusual punishment to humiliate and destroy the human spirit.

The first is written for the Guardian, but syndicated on Alternet. It describes Jose Padilla, an alleged terrorist.

The purpose of these measures appeared to be to sustain the regime under which he had lived for over three years: total sensory deprivation. He had been kept in a blacked-out cell, unable to see or hear anything beyond it. Most importantly, he had no human contact, except for being bounced off the walls from time to time by his interrogators. As a result, he appears to have lost his mind. I don’t mean this metaphorically. I mean that his mind is no longer there.

It simply must be read. Then, as if this weren’t enough, I read a similar article, but describing an entirely different case, in the New York Times. This time, it describes Donald Vance, a security contractor who was a whistle blower pointing out corruption within his organization in Iraq, who was then captured by the US Military and subjected to torture techniques because he had been “associated” with the organization whose flaws he had been instrumental in illuminating. The amazing part is that he took detailed notes of his stay, and the New York Times article presents this evidence along with testimony from Vance himself. It shows a detainee system so fundamentally broken and so insanely immoral that I had a lump in my throat while reading the words on the page. You should read them too.

The two men slept in their 9-by-9-foot cells on concrete slabs, with worn three-inch foam mats. With the fluorescent lights on and the temperature in the 50s, Mr. Vance said, “I paced myself to sleep, walking until I couldn’t anymore. I broke the straps on two pair of flip-flops.”

How will we ever redeem ourselves? There simply is no excuse for this kind of behavior. We have become what we sought to destroy.

The Humanity

Check out this interesting table about world views on torture over at the BBC.

Notice how the best scores are in Spain, France, and Italy (where more than 80% find the practice of human torture repugnant — what bleeding heart liberals! :)). I always knew that these countries had a better grasp of morality than Americans, but not by this degree.

Looking at those statistics for the US (where 38% think some degree of torture is permissible), it makes you think we’re not even a progressive country. Our percentage is lower than Turkey, the Ukraine… we’re only 1% off from Russia, for crying out loud! In Russia, they torture people “just in case” they do something bad!

The Three-Way Division of Web Development Labor

There is a great article posted by a developer of the open source FreeMarker project on a proposed “third” role for web development teams — something between a backend application developer and a frontend web designer.

Check it out.

I have to say, so much of this article rang true for me. I just finished a web application in which we had four developers but two main rules: backend developer and frontend designer/developer. Due to the type of project it was (and the skillsets of the people involved), backend work was done by 3 of the 4 team members, and frontend work was done by a sole designer.

The issues we had were that:

  • The backend developers could only make guarantees about what kind of object to pass to the view. But often, in order to get access to other “global” objects, the frontend designer would have to hack controller code as well. (Luckily, data services were tiered off out of the controllers, so at least the designer didn’t have to muck about with SQL.)
  • The designer hated the idea of pushing some logic “back” into application code. So, though we as backend developers proposed a layer for configuring things like which menus to display on each page, etc., the frontend designer preferred to engage in the copy-paste-modify antipattern instead, so that it could just “get done.” I can see why: it’s a pain to context switch from JSPs, HTML and CSS to Java POJOs that then will have to be checked using JSTL anyway!

If, early on, we had decided to use FreeMarker instead of JSTL, perhaps the frontend designer could have had access to more powerful macros, and wouldn’t have done the copy/pasting. But I think overall, the two tiers are too few; I agree that you need someone to sit in between and think about how to cleanly integrate backend logic and frontend design.

Linux Desktop Talk at NYU

I gave another talk for CANYU and the emerging open source clubs at NYU about Linux on the desktop. Here is the synopsis:

LINUX AND FREE/OPEN SOURCE SOFTWARE

The State of the F/OSS World Update
with talk/demo by Andrew Montalenti

December 5, 2006 @ 7pm
Room 813, Warren Weaver Hall

Open source software is now mainstream. Whether it’s the nearly ubiquitous Mozilla Firefox browser, the Azureus peer-to-peer client, the Eclipse IDE, or the Linux kernel, almost everything in the computer world has been touched by free / open source software developers collaborating across the globe.

Judging by the state of the community, this movement doesn’t seem to be losing steam. With Microsoft Windows Vista around the corner offering a potentially bloated and hardware-requirements-heavy experience, desktop Linux operating systems are taking aim at the big giant, with big support around Ubuntu, Fedora (Redhat), and SuSE (Novell), among others.

So, what’s next for the free / open source world? That’s what this talk is meant to help you find out. After explaining a bit of the history of free and open source software, and the history of recent community and corporate efforts to make it widely available, this talk will show off some of the new, cool technologies coming out of the open source community, such as 3D desktop effects, productivity tools, enhanced multimedia support, better support for laptops, and a full suite of industry-grade development tools. The talk will also discuss some of the legal and intellectual property issues facing the open source community, with a particular focus on the recent news coming from the Novell / Microsoft deal and Sun’s decision to open source Java.

Who is this talk meant for? Anyone who hasn’t tried out Linux on their desktop, or anyone who is at least mildly interested in the current and future state of the computer industry. Free / Open Source software has completely rocked the industry, changing every aspect of it from top to bottom, and this wave is only growing bigger every day. NYU students interested in copyright issues surrounding open source may also find this talk valuable.

In any event, this won’t be a boring lecture — it’s meant to be interactive and fun!

The speaker will be bringing free CDs of Ubuntu Linux, a community-driven desktop Linux operating system which you can install on almost any home PC! Come for the free CDs, stay for the revolution.

It went very well, with about 10 people in the audience. You can download the slides in OpenOffice or PDF. Admittedly, the slides aren’t as cool without the live demo of Beryl I did at the talk itself. Yay 3D desktop effects.