Outfoxed and trust networks, revisted

I think everyone should revisit Outfoxed, if you’re interested in a truly interesting new approach to bookmarking and web browsing that actually takes advantage of all this “Web 2.0” hype and nonsense.

I just mentioned this to Free Coders on the mailing list yesterday.

The application of “trust-based networks” is very wide. I think it’s the “trust” factor that makes eBay successful (perhaps even viable!). New services like Pandora and Last.fm recommend music by trusting that users who rate music are being honest. eBay users buy from trusted sellers by assuming that those who rate the sellers are honest. These assumptions may be fallible, but they’re better than nothing. And it’s only natural that this trend would spread to web browsing.

I think trust networks should be applied to political organizing, both to reduce risk of people showing up just to start trouble, and also to enhance the perceived value of a meeting based on the combined trust of its attendees. I’m going to think about this a bit more in the next few days.

(Meanwhile, I got in touch with Runar from the Google talk with Alex Martelli, who is working on an awesome library for Python called sqlstring. Check it out. I’ve also been thinking about expanding on my earlier ideas on Python inferred types; we’ll see if I find the time.)

Talk on Outsourcing

I recently gave a talk on outsourcing for Computer Advocacy @ NYU, entitled:

“Offshore Outsourcing: Roots in Corporate Power.”

It was meant to be an introduction to the subject, to precede the film screening we had of Greg Spotts’ “American Jobs.” I’ve posted the talk’s slides to my web server in SXI (27K) and PDF (212K) formats.

In the talk, I tried to show how outsourcing can be seen as stemming from the gradual ascendancy of corporate power in the world, beginning with the first laws enabling corporate personhood to today, when corporations pit governments against one another for who can provide the least humane economic regulatory system (which are then spun as “pro-business”–think, for example, of China’s inexistent environmental legislation, and how many high-pollution businesses have moved their shops there).

When corporations first gained rights as legal persons, they began to win cases in which they secured their right not to be regulated, and then began to win ideologues with a vision of the corporation which freely moves around the world, hiring all the labor it can find. Key to this vision, however, is that governments are helpless and defenseless–that they should not have the power to regulate corporations, since any such regulation creates an unfair situation in the global neoliberal “free market.” I try to make it clear that the end goal of this experiment is a global corporate state, in which labor laws and life/work balance simply doesn’t exist, as we all strive to be “more competetive” for corporations whose urge to lower cost will never disappear.

p.s. check out the book mentioned in my talk, Gangs of America by Ted Nace.

Got bored with Descartes and Spinoza, wrote a patch to powernowd

While I was studying, I noticed that I wasn’t exactly happy with the cooling/cpu frequency scaling on my laptop. I use a nice program called powernowd which scales my CPU speed up and down depending on various factors related to system load. But I didn’t like how my setup was kind of “all or nothing.” When I am plugged into AC, I switch to “performance” mode which just runs me at 100% CPU frequency all the time (making my laptop hot, my fan noisy, but my machine fast), whereas when I’m unplugged I switch to “userspace” mode, which lets powernowd kick in, and he jumps about from 400mhz to the full 1.6Ghz based on load, keeping the machine cool but also making it feel a bit sluggish since if I’m overloading my CPU at 400mhz it’s already “too late” to pump it up, it will have already felt slow for at least an instant.

So I have this conflict: hot and responsive, or cool and sluggish. I thought, well, I must be able to come to a compromise.

I decided to take a look at powernowd’s code, and it turns out it’s written quite straightforwardly. Within 30 minutes of tinkering, I had a patch that did what I wanted. With another 30 minutes, I polished it and made it quite commitable.

Basically, I added a new mode called “COOLING” to powernowd, which runs your CPU a few notches below your full frequency (which I call your “cool_spot”), based on the following approach:

  • if you have two frequencies available, you normally run with the lowest.
  • if you have three frequencies available, you normally run with the second from highest.
  • if you have four frequencies available, you normally run with the third from highest.
  • if you have five or more frequencies available, you normally run with the fourth from highest.
  • if your load goes above your specified trigger (“highwater” in the code), you jump to highest frequency. When it lowers (“lowwater”), you go down to your cool_spot, but not below it.

On my machine, I have 5 frequencies (1.6Ghz, 1.5Ghz, 1.4Ghz, 1.2Ghz, and 400Mhz), and so I normally am running at 1.2Ghz. This new COOLING mode runs while I’m plugged in, and keeps my machine nice and cool but still lets it immediately respond when I want to do something, like a workspace switch.

I then hacked the init.d script to have a BATTERY and AC mode, and switch between AGGRESSIVE and COOLING modes accordingly. Now, when I’m unplugged, I get the best battery life and pretty good performance, and when I’m plugged in I get a cool notebook with good performance.

I’ll probably post the patch after my midterms…

Development under Windows: why so painful?

It’s really weird. Lately, I’ve been doing so much development in a *nix environment, that doing the development in Windows is really painful for me. I don’t have any of my good old UNIX tools, I don’t have hotkey-optimized user interfaces, I don’t have speed and control. But more than anything else, I don’t feel like I know what’s going on under the hood.

Today, to take a break from reading Philosophy, I decided to work a bit on this little Java Servlet project I’ve been hacking on. (Will be “released” later.) At some point this past summer, I decided to remove Linux from my main desktop machine and just consolidate all my Linux data onto one machine–this made my life easier so I didn’t have three total (one Windows, two Linuxes) places where my shit could be. But the sacrifice is that my laptop screen is small, so sometimes I want to develop with a big screen and thus want to use my desktop.

Web development, especially, makes sense for me under Windows, since I’m comfortable with the major graphic and web design tools (Photoshop, Dreamweaver, Illustrator) and don’t think the Linux “equivalents” (GIMP, Bluefish, Inkscape) are good enough.

But I decided–may as well have the code open on Windows too, since it’s not C hacking I’m doing, but Java. So I installed Eclipse, and the J2EE, and got cracking.

But under Windows, there are all sorts of gotchas. When my UNIX tool craving gets really bad, I need to drop into cygwin, which isn’t so bad. But without good workspace switching (I have VirtuaWin, but it kinda sucks), and without a customizable window manager, I am really much slower. But here’s the other weird thing I ran into. After awhile of coding, I realized that Eclipse wasn’t reading my JavaDoc information for the JDK (no cool descriptions in my autocomplete tooltips). So I go snooping around the preferences file and can’t find anything, I enable a billion options but no luck. But then, eventually, I realize that it’s very possible Eclipse is using a different JDK. In fact, I look in the dialog, and Eclipse is using some J2SE environment that some other application installed, not the J2EE I installed right before Eclipse. And that J2SE is missing the Java API source code and JavaDoc comments.

The reason this seemed so non-obvious to me is because I’m not used to systems which are completely fucking disorganized. Say what you will about Linux not being user friendly, but, by God, you won’t find it likely to find two different JDKs installed on my machine, and even if you do, only one will be getting used (thanks to Debian’s “alternatives” system). Every application on Windows statically compiles, includes its own libraries, and spews its shit all over the file system and registry. No database tracks it, so your system is a fucking nightmare.

I couldn’t even do a reasonable search to find the JDK I needed, either. It turns out it was in C:\Program Files\Sun\j2sdk1.4_02, which may not sound so bad, but considering on Linux I just think, “Where are libraries stored? /usr/lib” and then in there I think, “What is what I’m looking for called? j2sdk” I quickly find any Java environments in /usr/lib/j2sdk1.x-sun.

On my Linux system, which has not that much installed, I just ran a du -hs /usr/lib/ and got 1.7GB. That means on my relatively lightly-loaded Linux system, 1.7GB of raw 0s and 1s are sitting there waiting to be used as SHARED libraries. Meanwhile, on Windows, there could be any amount of duplication of the equivalent libraries, floating around in various Program Files directories.

I can’t believe there are acutally some Linux critics that believe we should be going in this direction, eliminating things like emerge, apt, and rpm and instead just have statically-compiled binaries that come with their own binary libraries and have users duplicating this stuff across their system. Not only is it insane from the point of view of giving control to the user, but it’s also just plain wasteful.

Just For Fun: The Story of Linus Torvalds

For the last couple of weeks, my bedside reading has been this half-biography, half-autobiography on Linus Torvalds. I have to say, however, that the book is like two books mixed into one. Chapters alternate between Linus talking about his life and about big moments in Linux’s history to David Diamond describing modern-day Linus with a kind of forced wonder. Truthfully, Diamond comes off as a sycophant who could care less about Linus’s flaws and positive characteristics, and cares more about molding some kind of “image” of Linus as containing a humility and genius simultaneously. Near the end, I started only skimming the chapters not written by Linus. Diamond’s really not a good writer, either. (Sorry Dave.)

Truthfully, the book kind of pops the lid off Linux and makes you understand it as much less glamorous than say Wired Magazine described it to the public. Linus really just talks about not having a social life, sitting in his room with curtains covering his window, coding all day. Not exactly the ideal role model, I think. Don’t get me wrong, I love the Linux kernel (as much as one can love imperfect software), and Linus made a great contribution toward keeping the UNIX world and UNIX principles alive, but it’s just that I like to think of open source developers as something other than the stereotypical, introverted geek. In fact, much of Linus’s chapters is devoted to his apprehension about giving a public talk about Linux. When I think about the fact that I’ve given three or four of them to date, and enjoy it more every time, I see how different I am from this kind of stereotypical geek.

It also kind of made me dislike Linus. When I saw Revolution OS (a DVD on the rise of open source), the movie kind of endeared me to Linus’s practical nature as opposed to Richard Stallman’s religious idealism. I like idealism, but Stallman is really religious about it. And he’s bitter. Linus, on the other hand, has that great Northern European, “I’m just gonna go with the flow” attitude.

But this book made me realize that Linus is religious is his own sort of way. Included in the book is Linus’s flame war with Andy Tanenbaum on monolithic versus microkernel designs. Truthfully, I’ve studied operating systems and I’m not even sure which design is best, and Linus makes a decent argument of why microkernels end up being just as complex, or more complex than monolithic ones. But what I didn’t like is that in the flamefest, Tanenbaum said that deficiencies in MINIX were due to it being a hobby, and that he had duties as a professor. Linus responded, “Re 2: your job is being a professor and researcher: That’s one hell of a good excuse for some of the brain-damages of minix. I can only hope (and assume) that Amoeba [Tanenbaum’s future OS project] doesn’t suck like minix does.”

This just shows me that Linus really is an asshole sometimes. He states this outright in his book. So now, truthfully, I may like the open source movement, but I think I “at least dislike” two of its most major players (Torvalds and Stallman).

Finally, I think a clip from Tanenbaum’s website points out a nice principal in OS design:

Also, Linus and I are not “enemies” or anything like that. I met him once and he seemed like a nice friendly, smart guy. My only regret is that he didn’t develop Linux based on the microkernel technology of MINIX. With all the security problems Windows has now, it is increasingly obvious to everyone that tiny microkernels, like that of MINIX, are a better base for operating systems than huge monolithic systems. Linux has been the victim of fewer attacks than Windows because (1) it actually is more secure, but also (2) most attackers think hitting Windows offers a bigger bang for the buck so Windows simply gets attacked more. As I did 20 years ago, I still fervently believe that the only way to make software secure, reliable, and fast is to make it small. Fight Features.

I agree. But does a microkernel design actually reduce the overall size of the operating system, or does it just reduce the size of whatever you consider to be the “microkernel”? That is, just because a file system is implemented as a file system daemon talking to a driver subsystem through message passing doesn’t necessarily mean the file system, or driver subsystem, are secure. Insecurity could exist even at the boundaries, no? Not to mention instability.

I think Linus and Tanenbaum have to agree that this debate isn’t an open and shut case. The best kernel is probably one that mixes modularity, a strong kernel/userspace boundary, and some of the fancier features of a microkernel approach, while not sacrificing elegance of design or performance.

Free Coders at NYU

I’m organizing a group of people interested in hacking open source software in a team environment. Right now I’m calling it Free Coders at NYU, and have already set up a wiki and mailing list. This could end up being very cool. Next meeting is hopefully this coming Tuesday.

I set up a mailing list with GNU Mailman (link above), which was decently painless under Debian Sarge. The only annoying thing was utilizing my virtual e-mail address mappings which are stored in MySQL, but I figured out a trick for that.

I’ve already spoken, via e-mail, with an open source developer who works on gstreamer among other projects, Ronald S. Bultje. He has already tentatively agreed to do a talk for us sometime this year.

N-way parallel mail retrieval with getmail and bash

I wrote a pretty sweet script tonight. It parallelizes the getmail retrieval process, while still printing prefixes so I know which accounts download which messages. This means that instead of my mail fetching process taking sum(i1,…,in), where i is the length of time for a given mail retrieval, my fetching process now takes max(i1,…,in).


#!/bin/sh
GETMAIL='python2.3 -Wignore /usr/bin/getmail'
unwanted() {
  grep -E -v '(Copyright|getmail|Simple)';
}

echo "N-WAY GETMAIL RETRIEVER SCRIPT:"
$GETMAIL \
  --rcfile=/etc/getmail/account1 \
  2>&1 | sed -e \
  's/.*/account1................: &/g' \
  | unwanted &

$GETMAIL \
  --rcfile=/etc/getmail/account2 \
  2>&1 | sed -e \
  's/.*/account2................: &/g' \
  | unwanted &

...

$GETMAIL \
  --rcfile=/etc/getmail/accountN \
  2>&1 | sed -e \
  's/.*/accountN................: &/g' \
  | unwanted &

wait

PIDA: Python Integrated Development Application

PIDA 0.2.2 was released recently. This is truly a novel development in the Python/OSS world. What PIDA provides is a nice plugin system and the “makings” of an IDE. So, in a nice IDE you have a class browser, an integrated debugger, a profiler, maybe even a RAD-like GUI builder, an interpreter console, etc. The one piece that tends to be most controversial in every IDE is the text editor. This is one-part because UNIX people are really crazy about their text editors, but two-parts because text editors are very important programmer tools, and no one wants to learn a different text editor for every language one uses.

vim happens to be awesome for C programming, which is probably why a lot of UNIX hackers use it. But for Python, more advanced support would be nice. PIDA can run and connect to a vim server instance in order to allow you to have an “add-on IDE” for vim.

But even more interesting to me is the culebra plugin, which provides a code-completion-savvy GtkSourceView inheritor, which has the initial support for fancy Intellisense-like features.

I’ve already spoken to the developers of PIDA, and they said they would very much be interested in seeing Python Intellisense features brought to VIM. When I started thinking about different approaches to doing this, I realized that the whole OSS community could benefit from a general Python module that enhances the Python introspection features (and perhaps combines them with source code parsing) to make available nice productivity-enhancing features. I was thinking of calling this beast “Pyductivity.”

More on that later. For now, check out PIDA.

Exa: a new architecture for Xorg

This is exciting news. A Trolltech developer has modified KAA, the acceleration architecture used in Keith Packard’s experimental “kdrive” Xserver, to work with the traditional Xorg tree. He announced this new development in an e-mail that makes it clear it is extremely easy to get drivers to use Exa to gain Apple/Windows-like graphics performance.

I know that the unichrome project generally doesn’t bother itself with these very desktop-oriented features (their focuses are more on MediaPCs, etc.), but I think this may be an excellent way for me to begin hacking the new modular Xorg tree I mentioned last time. If I added Exa support to the unichrome driver, would that mean transparency and full-on graphics acceleration for my X desktop, what I’ve long been waiting for? We’ll see.