Andrew Montalenti

So, one of my employers ended up with a fried hard disk, for the second time in a row. The main reason is that the PC this HD is contained in sits in a corner with little-to-no airflow.

In order to recover the drive, I am actually taking a different approach from my last recovery effort, mainly by necessity. This disk is seriously damaged–lots of bad sectors, and its partitions are not readable by any NTFS driver, be it Microsoft’s or the open source one. This makes simply using the wonderful R-Studio tool I used last time currently impossible, due to the fact that it won’t even see the drive properly within Windows, and will hang all over the place.

Indeed, what I needed to do is drop down a layer of abstraction: away from filesystems, and into blocks and sectors. Unfortunately, in the Windows world this drop down is difficult, so I had to use my Linux laptop to make this jump.

I found a wonderful tool to help me out called dd_rescue, which is basically a dd with the added features of continuing on error, allowing one to specify a starting position in the in/out files, and the ability to run a copy in reverse. These features allow one to really work around bad sectors and even damaged disk hardware to get as much data as possible out.

Unfortunately, the use of this tool was encumbered by my laptop’s relatively simple bus design. Apparently, if I stuck two devices on my USB bus (like two HDs I was using for this process), the bus would slow to a crawl, and the copy would move along at an unbearble 100kB/sec. I tried utilizing firewire and USB together, but got only marginal improvements. What befuddles me is that in the end, the fastest combination I could come up with is reading from the Firewire enclosure with my laptop and writing to the firewire enclosure of my desktop across the LAN utilizing Samba. Very strange indeed. Now my performance is more like 6MB/sec, factoring in all the breaks dd_rescue takes when it encounters errors. I have 6GB of the more critical partition written, but it’ll probably take a couple hours to have a big enough chunk that I can test R-Studio’s recovery of it.

The only reason I’m even writing about this is because I find it hilarious how many layers of abstraction I am breaking through to do a relatively low-level operation. Think about it:

1. My broken IDE drive is converted to Firewire by a Firewire-IDE bridge.
2. My Firewire PCMCIA adapter is allowing my notebook to take in that connection.
3. The Linux kernel is allowing firewire to be accessed via various ieee1394 ohci drivers.
4. The Linux kernel is abstracting the firewire disk as a SCSI disk, using emulation.
5. The SCSI disk is being read by dd_rescue and written to a file, which exists in the path /mnt/smb/image/sdb5.
6. That path seems local, but is actually a mount point. That mount point seems physical but is actually handled by a Samba driver.
7. The writes by dd_rescue to that image file are being sent through the kernel’s TCP/IP stack, and flying through my switch, and being accepted by Windows XP’s network stack.
8. Windows XP is writing that data to an NTFS drive, which is itself connected by a Firewire-IDE bridge (and therefore all the above steps’ equivalents for Windows apply).

I am surprised with that many layers, that this copy is even working. I really should have just taken a machine apart and connected these drives directly by IDE, to save myself a few layers.

I was just looking at my strange contact page, where I list my e-mail address using a sort of obfuscated string with _ and * characters mixed in. And then I saw someone’s e-mail address listed on the web with the following format:

user () domain ! com.

At that point, I started to think about all the other variations of this spam-protection trend I’ve seen, like user ///at\\\ domain ///dot\\\ com, and I realized that many of us are taking the wrong approach. Myself included. For example, the one above could easily be found by knowing the common TLDs and working backwards from there. If I find a “com”, “org” or “net,” and then look at the string tokens which occur before, I can assume any string of valid characters (say, alphanumeric characters) which is followed by whitespace or invalid chars (like parentheses and exclamation points) can be taken as a valid part of the address. From there, we can easily split user () domain ! com into its proper parts, and construct the e-mail. This same approach works for say, user ///at\\\ domain ///dot\\\ com.

So what I realized is perhaps it would be better to insert other e-mail addresses in there that might get picked up as part of an e-mail address, even in a heuristic scan. For example,

user __at__ domain :: NOT [email protected] :: __dot__ com

That seems more secure to me 😉 Another approach is just to prevent the TLD from being a complete token. This is the approach I took. Turn com into c_o__m or something, and you’re less likely to get picked up in a scan that is searching for “com”.

This /. article has responses from Microsoft Linux Lab manager Bill Hilf. I responded to this post from a Microsoft employee. My response follows. Continue reading Microsoft’s anti-competitive behavior

I wrote a pretty sweet script tonight. It parallelizes the getmail retrieval process, while still printing prefixes so I know which accounts download which messages. This means that instead of my mail fetching process taking sum(i₁,…,i_n), where i is the length of time for a given mail retrieval, my fetching process now takes max(i₁,…,i_n).

#!/bin/sh
GETMAIL='python2.3 -Wignore /usr/bin/getmail'
unwanted() {
  grep -E -v '(Copyright|getmail|Simple)';
}

echo "N-WAY GETMAIL RETRIEVER SCRIPT:"
$GETMAIL \
  --rcfile=/etc/getmail/account1 \
  2>&1 | sed -e \
  's/.*/account1................: &/g' \
  | unwanted &

$GETMAIL \
  --rcfile=/etc/getmail/account2 \
  2>&1 | sed -e \
  's/.*/account2................: &/g' \
  | unwanted &

...

$GETMAIL \
  --rcfile=/etc/getmail/accountN \
  2>&1 | sed -e \
  's/.*/accountN................: &/g' \
  | unwanted &

wait

This is why I love Wikipedia.

PIDA 0.2.2 was released recently. This is truly a novel development in the Python/OSS world. What PIDA provides is a nice plugin system and the “makings” of an IDE. So, in a nice IDE you have a class browser, an integrated debugger, a profiler, maybe even a RAD-like GUI builder, an interpreter console, etc. The one piece that tends to be most controversial in every IDE is the text editor. This is one-part because UNIX people are really crazy about their text editors, but two-parts because text editors are very important programmer tools, and no one wants to learn a different text editor for every language one uses.

vim happens to be awesome for C programming, which is probably why a lot of UNIX hackers use it. But for Python, more advanced support would be nice. PIDA can run and connect to a vim server instance in order to allow you to have an “add-on IDE” for vim.

But even more interesting to me is the culebra plugin, which provides a code-completion-savvy GtkSourceView inheritor, which has the initial support for fancy Intellisense-like features.

I’ve already spoken to the developers of PIDA, and they said they would very much be interested in seeing Python Intellisense features brought to VIM. When I started thinking about different approaches to doing this, I realized that the whole OSS community could benefit from a general Python module that enhances the Python introspection features (and perhaps combines them with source code parsing) to make available nice productivity-enhancing features. I was thinking of calling this beast “Pyductivity.”

More on that later. For now, check out PIDA.

This is exciting news. A Trolltech developer has modified KAA, the acceleration architecture used in Keith Packard’s experimental “kdrive” Xserver, to work with the traditional Xorg tree. He announced this new development in an e-mail that makes it clear it is extremely easy to get drivers to use Exa to gain Apple/Windows-like graphics performance.

I know that the unichrome project generally doesn’t bother itself with these very desktop-oriented features (their focuses are more on MediaPCs, etc.), but I think this may be an excellent way for me to begin hacking the new modular Xorg tree I mentioned last time. If I added Exa support to the unichrome driver, would that mean transparency and full-on graphics acceleration for my X desktop, what I’ve long been waiting for? We’ll see.