When I was about 10 years old I was in a math class that periodically (once a week, I think) took place in a computer lab. I learned to program in BASIC and to read and write from a cassette tape. A little later I learned to read and write data from a file on a floppy disk or hard disk in BASIC, Pascal, and C. I learned that sometimes this can be really slow if record size varies, so I learned some ways to make that go faster. When I was 24 I learned about DBM files which do a lot to hide that complexity from you so you can just look something up by ID and get the thing really fast. After working with that for a while, I moved up to a really crappy SQL database called mSQL that had lots of flexibility but was really good at corrupting your data. Shortly thereafter I started to use Sybase and Informix databases and finally learned what the big deal was with this 'transactions' stuff. Then I got to work with Oracle, Microsoft SQL Server, and a few others that all basically achieve that level of flexibility and feature richness, with the associated costs of tricky backup and large installation and runtime footprint. And here and there I saw oddities like embedded databases and object databases. I was lucky enough to see most of this in ascending order of complexity and size of the problem solved by each approach. Printing "Hello World" to a text file is way easier than setting up Oracle, but Oracle does a tremendous amount of stuff to make sure that your "Hello World" is easy to find and never goes away or gets hosed unless you say so.
It is in this context that I consider myself a "database guy". I've been working for 10 years on web applications for businesses, and most of them have a set of problems that fits pretty squarely with what ACID-compliant relational database servers (Oracle, Postgres, MS SQL Server, etc.) provide. That's not always the case, and both of my current contracts involve storage in both a relational database and a different persistence mechanism. In one case it's a bunch of text config files; in the other it's a simple record storage mechanism built into the PalmOS. I like to think that I know when it's time to just tell the people who write checks to buy something big and sophisticated, when it's time to find an off the shelf free library to manage stuff on disk in a fairly simple way, and when it's time to roll your own teeny weeny storage system.
So today I had a reason to want to look at the Firefox browser history file on my laptop. For an unimportant reason I wanted to know at what time I was looking at something a week and a half ago. Firefox lumps all visits over 6 days old into a single big list of URLs, which is pretty worthless. The timestamp is known, but you can't have it through the UI, nanny nanny boo boo. OK fine, I'll just look in the history file and...
Oh my god.
It's called Mork format and it is an abomination before god, a blight, a sin against all that is good and holy in software engineering and computer science. Don't take it from me; the guy who wrote the code it replaced had this to say:
"It is impossible for non-Mozilla programs to extract data from ~/.mozilla/*/*.slt/history.dat because it uses Mork, which is — and I do not use these words lightly — the single most braindamaged file format that I have ever seen in my nineteen year career."
See also this thread entitled "when the database worms eat into your brain" about trying to get data out of it as if it were a file format you could make sense of. Here's the primer which was apparently written by a Martian who has a grammar and spell checker but whose brain is installed in his head backwards compared to earthlings. I still don't understand what he's trying to say in the second paragraph, and I don't have enough Advil left after my ankle injury back in November to keep trying.
So, the file itself is unreadable, the docs are byzantine also, the author's resume screams Rain Man, and his web site looks like a lipsum generator and some kind of English text randomizer used by spammers to confuse Bayesian filters spliced their word spewing genes with the circular-button obsession of VisualAge. Whew!
Okay, so at some point this guy wrote code that worked, and still works, and is in the code today. Based on the above thread and stuff it links to, this is mainly because the code and the format are so spectacularly complicated and hard to understand that no one dares touch it. But he made it work, and somehow defeated any code reviewing or architecture reviewing bodies within Netscape, including Zawinski trying in vain to get the guy to think pragmatically.
The big gaffes are that it exists at all, and what a total design disaster it is. But the implementation is pretty scary too: have a peek at some of the source code. The odd thing is that the code seems to be reasonably clean at a microscopic level, probably conforming with whatever coding standards were applicable. Let's look past the style to what it actually says and does. Hold fast to your sanity becuase madness lies herein. morkBeadMap? Atoms, Beads, Blobs, Cells, Cursors, Deques, Handles, Maps, Nodes, Pools, Ports, Probes, Sinks, Spaces, Tables, Thumbs, Yarns (my favorite), and Zones — whatever the hell all of those are exactly — to store fucking address book entries and a list of URLs. 10 lines of comment for a function that does nothing and then returns the result code for "OK", no wait even better, 21 lines of comment for one that returns the result code for "not implemented". As if, in his mind, this was just a placeholder implementation of a much more sophisticated API, to be replaced later with something much bigger.
Microsoft bundled IIS with NT 4 to cut off Netscape's air supply, and they basically succeeded in crippling Netscape that way, but they didn't make Netscape 4 suck. Netscape did that. This code got through Netscape's development organization and shipped, and it's still in Mozilla, Firefox, and Thunderbird. If you've done any web design you know what a flaming bag of dog shit on the web developer's doorstep Netscape 4.x was, and now we have a nice case study of the broken development organization that produced it. Working code (which jwz says was fast and space-efficient, and I have no reason not to believe him) was tossed out and replaced with this ludicrously overengineered beast, written in house from scratch by one guy and understood by him and him alone despite his failed attempts to explain how it works and why it works the way it does.
It's tempting to look at this and want to violate the Things You Should Never Do advice to never rewrite from scratch, but hang on for a second. Joel says they did rewrite the browser from scratch, but this code remained. Of all the things to not throw out... ugh.
An interesting side note: this developer probably could kick ass at the typical tech interview question that aims to find out if your brain can solve computer science problems. But he totally blew a software engineering challenge, which was to figure out requirements and design an appropriate solution, and he also blew the organizational intelligence question of listening to somebody who already had attacked the same problem and solved it in a completely reasonable way. (That is, the same problem assuming you don't take twenty steps back and view all data storage as a problem to be solved with some daring new ultra-complex code that you've been meaning to write.)
Funny: SNL: Always Be Cobbling (Put that cocoa down! Cocoa is for cobblers only.)
Enplaned is making me an airplane industry business analysis fanboy. As if that weren't enough, the author is also on the side of the fence I am on, in which the letters and symbols and words and grammatical constructs you use when writing are actually considered to be important in communicating one's point:
...the word you want in reference to Virgin's reputation is not "cache" (a place in which to store things) but "cachet" (prestige).
(from Boyd on Virgin America) This is opposed to the crowd, featured in places such as Slashdot, that believe that spelling and grammar and using a word that actually means what you think it means are all a waste of time and that "aw you know what I mean" excuses any amount of linguistic sloppiness. (I find that attitude highly ironic in the Slashdot crowd since its members typically spend all day crafting precise source code and text configuration files for extremely unforgiving computers to interpret. Shouldn't they be more precise than most people when writing?)
Maybe someone else will manage to get techies to stop pronouncing cache, which is a homonym with cash, both rhyming with trash, to stop pronouncing it as a homonym with cachet, both rhyming with sashay. Techies spend so much time reading that we learn lots of big words, but few of us actually bother to look up pronunciation in a dictionary so we don't say nonsensical things like 'it then stores the data in its cachet' in public.
Is there anything more badass than skydiving from an altitude of 30 kilometers? I think that's the stratosphere. That's some altitude, man. Damn. That's 1/10 of the orbital altitude of the Space Shuttle. Could an astronaut jump off of the space shuttle with a parachute and oxygen tanks and survive the drop, or would re-entry burn him up?
Adam B. pointed out that the bandwidth tester I used from my laptop to measure the speed of my dialup connection over bluetooth over a Treo 650 cell phone data connection was probably wrong. 1xRTT technology, which is what the Treo (and my previous, much less fancy Samsung VGA1000 phone) has for data connectivity, doesn't do 600kbps. This MAKE magazine article has bandwidth graphs of someone else's tests that seem more realistic, and more in line with the level of responsiveness that I experienced.
That said, it was still badass to be able to set my laptop near my phone, dial up through the phone, and then share my connection via wi-fi so that Kim could get to the internet as well. On top of that we were both using Hamachi to get to our email. Sweet. Of course since we were traveling I hooked all the cables up to make sure all the batteries were charging for the long trip home. Eventually, as many Treo owners have mentioned, the USB cable came unplugged from the Treo and its battery ran down, and for a while it was so drained that it refused to do anything other than charge. The folks who complain about the cable not staying plugged in were right. But while it was all working, it was pretty darn cool. I got some work done, settled a bet (definition of pony vs. colt), and downloaded the X-Men 3 teaser trailer. :)
I don't get why anybody likes MySpace. It's slow as hell, everybody dresses it up with horrible animated eyesore designs and background music that sucks, and for some reason musicians think that they need to have a presence there. WTF? Half the time I've gone there and either the page or MySpace itself has been broken in some way.
Although I do recommend that musicians avoid flash for their web sites, I found a pretty cool flash MP3 player that you can embed in a page if you don't want to rely on someone's browser to know what to do with an MP3.
Last week Kim's dad was in town and we were looking for a restaurant we had been meaning to visit. It's run by the same people as Golden Flower. Google "golden flower" "san francisco", and the results page has a pretty funny first hit. (NSFW!) Not a restaurant. Nope. (We ended up going to Bacco anyway, which was outstanding.)
Aww yeah. I've achieved the holy grail (for me, anyway) of road warrior internet access: getting my laptop to connect via Bluetooth dial up networking through my new Treo 650, with a VPN running over it for security that lets me just mount Samba shares from my home file server, run IMAP in the clear to a firewalled mail server, etc. Sweet!
The process goes like this: Update the Treo to firmware version 1.12. Turn on Dial-Up Networking in the Bluetooth manager on the Treo. Pair the palm and the Mac. When the pairing process is complete, the wizard will ask if you want to dial up through this device, and you say yes. Then you just do the stuff in Trevor's instructions (Sprint PCS Vision username and password, #777, use "Sprint PCS Vision" as the modem device), and dial up!
The latency is pretty high but the bandwidth is on the order of 600 kilobits/sec. The best part though is that the laptop and the phone are just sitting near each other. There's no flimsy USB cable that disconnects you if you wiggle it slightly, as was the case with my last phone. I'm way out on the far end of Long Island, NY right now and even though I have one and a half signal bars on my phone, I'm online faster than dialup and near DSL speed. Yee haw!
I'm back to a normal sleep schedule now but this is still pretty funny.
I regularly butcher the English language for [what I consider to be] comedic effect, probably because of sniglets. I try to balance this with a great deal of care given to spelling and grammar when I'm writing or speaking seriously. Mike attempts to bust me for saying "I've graduated" after an earlier entry I made about how annoying it was that people were misusing the verb "to graduate" in the incorrect form "I graduated high school". He misses, though, because of what comes after the word "graduated" when I used it: a period. Intransitive. See answers.com's entry for graduated. "I graduated high school" = incorrect, unless your job is to rate high schools, or to landscape them. "I graduated from high school" = correct. "I was in high school and stopped going because I graduated" = correct, at least according to Webster's.
Yesterday when running
Oof. I changed nothing. It's worked dozens of times. Now, it fails in a spectacularly strange way. I know what a public key is, and I like that the software that I run has been cryptographically signed by the people I get it from, so I know who it came from. I get what the message is saying. I just have no idea why it broke or how to fix it, since this is something that until now Just Worked(tm).
Google to the rescue: Debian Testing apt-get update Breaks, NO_PUBKEY. That wget command line in the linked page fixed it. All better now. I still think I should never have seen this problem, but maybe there's some reason that I should have to use a non-secure, non-signed way to update the most critical element of the package system's security architecture. IMO, it shoulda just been grabbed a few months ago with a prior update, and been used automagically.
In the process of trying to fix this, I changed my Debian mirror server to http.us.debian.org. My sources.list entries look like
I set up Hamachi yesterday, emboldened by the Security Now! podcast endorsement by Steve Gibson, and I must agree that this thing rocks. It's easy and it's secure, which is very rare. I've set up IPSec, PPTP, more SSH tunnels than I care to remember, and I started to set up OpenVPN yesterday (which is why I was doing
Funny: Caution: Cat Vomit sign.
(Updated: link fixed to Kerry Skeen story below)
For the past couple of weeks I've been nocturnal: up at 6pm, work all night, go to bed at noon. It started as an all-nighter, because I was working on something complicated and didn't want to stop until it was done. That took until about 9am, and that pretty much threw my sleep schedule out of whack. My co-workers are getting irritated that I'm never around when they need to talk to me, which is understandable. It's kind of worked out so far with Kim because she sort of has the house to herself during the afternoon, but there's overlap where we're both awake at both ends. The big problem with this is that we're going to be visiting family this weekend, so I'm pushing my schedule back bit by bit to try and fix it. Yesterday I went to bed at about 2pm and I got up at about midnight.
The funny part of all this is that it's not unusual for me to be awake and working at 2am, and to have 3 or 4 co-workers be awake and also working (I can tell because of IM presence on Yahoo and AIM as well as our private IRC server). Someone that Jason and I used to work with at Viant started working with us recently. Day 1, 4:30am, he's working. He's one of us already!
When I'm not coding like a vampire, one of the things I've been doing is listening to the Ricky Gervais Podcast, which is hilarious. If you like to hear sarcastic British comedians making fun of monkey-obsessed rubes who believe every outlandish story they hear, you'll love it. For me, it makes doing the dishes, laundry, etc. a fun time. I can't listen to spoken word stuff like podcasts when working; I guess it's the verbal center of my brain that can only deal with reading the screen or hearing someone talk, but not both. So it's fun to have a chore that's nonverbal since that allows me to listen to the podcasts. There's an advertisement for you: Introducing Podcasts! They'll keep your sink empty and your favorite clothes clean! Your kitchen counters will be spotless and your trash will always be at the curb!
Interesting: Kerry Skeen: the Ego That Ate Independence Air, via Faisal: The Man Who Said No to Wal-Mart, Overstock shares fade as CEO warns of 'drugs or dead body' caper, Chairman of the Joint Chiefs of Staff says Rep. John Murtha's comments are "damaging to recruiting". Well, duh, when a man famous as a hawk says that he wouldn't join today's military, that says something that prospective enlistees will hear. But is it wrong to say that? There's a story about this situation. I think it has something to do with an emperor and some clothes. Yeah. ("Ssh, don't tell anybody that the war in Iraq isn't going well! Nobody knows that unless you tell them, and it'll be fine if we just keep pretending! Ssh...")
Maybe you can help me interpret this dream I had last night. Here goes:
I'm back in high school, 'cept I've graduated. For some reason I have to participate, all dressed up spiffy, in the current graduation ceremony. I can't get out of it. I don't want to be there but I have to. Please don't ask me what leverage was used to coerce me to be there, I don't know what it could possibly have been. Just stuck there, OK? And it's a two day saga pageant, outdoors, indoors, so many different events, tons of people. Right now I'm at the part where there's an outdoor concert featuring some mediocre local band fronted by a short hispanic dude in a wife beater playing a guitar and singing. But it's no ordinary guitar, because it's me doing the dreaming. Oh no, it's an acoustic 12-string bass guitar, an instrument I didn't think existed (in acoustic form) until just before I typed this sentence and found this excellent example. Short scale, though, same size as an acoustic guitar, just with three strings where each string on a 4-string instrument would go. Sounds killer. Big ups to this guy for wielding this savage axe. Except his instrument is out of tune, which sounds really bad. It burns us. He notices and stops playing mid-tune, hops offstage and wanders over to a little table where he starts retuning. What? The rest of the band is as surprised as I am and they kinda stop as the song falls apart. Weak! Musicianship rule 1, keep playin'. They blew it. Doesn't this guy know you're supposed to tune before you play? To keep playing till the end of the song, or to try and fix the tuning a bit during a spare moment and then keep going? One definitely doesn't just abort in mid-song.
At this time I turn to the guy next to me, who of course is The Rock, and he returns my look of disbelief. We both shrug and laugh, just as Lenny Kravitz arrives sits down on The Rock's other side.
At this time the telephone woke me up and the dream was over, and that was fine because I had no desire to kill more time watching this guy and his 12-string guitar while waiting for the graduation to crawl along. Some people fly or see dead friends in dreams. Apparently I watch boring music in boring places with The Rock.
Thank you O shuffle mode for bringing back one of my forgotten favorites, Control and Resistance. ProgArchives calls it "Essential: a masterpiece of progressive music". If you hate speed metal, skip it, you'll hate this too. If you like Queensryche and Dream Theater, get this album now. Witness this gushing Metal Observer review in which the reviewer nails it: tune out the screamy vocals (an interview I read said that the singer hated singing like that but the band basically forced him to because they all liked that style!) and pay attention to the crazy complex fast intense music. Guitarist Ron Jarzombek does indeed manage to straddle the indulgent virtuosic shreddy guitar worlds of speed metal and jazz fusion, and really the arrangements are so complicated that it's really hard not to mention death metal and "experimental post-metal" too. (Not that I would know anything about experimental post-metal from personal experience.) If you also look at this intricate dense busy artwork while listening, your head may explode, or maybe you'll have a seizure. So don't.
I think it's interesting that now there is a precedent that $7.50 is a fair price for taking over someone's computer via a rootkit. If I ever get arrested for breaking into somebody's system (which assumes that I would first have to want to do that) I'll offer to pay the $7.50 fine to avoid jail time. Hey, it's much cheaper than Sun's $1/CPU/hour. $7.50/PC until you get caught! And as Bruce Schneier notes, "The story to pay attention to here is the collusion between big media companies who try to control what we do on our computers and computer-security companies who are supposed to be protecting us." So, you won't get caught for a long, long time because McAfee and Norton and all the spyware programs looked the other way because it was Sony who installed the rootkit (or they just plain missed it, which is pitiful since this is *exactly* the sort of thing that you install those programs to protect you from). Hmm, maybe Sony is doing this on purpose. After all, the PC and hard disk and CD and MP3 player people have been making money off of digital music, and the RIAA hates them for it. Maybe this is their way of undermining computer hardware sales: provide a backdoor to the existing grid of internet connected PCs, by installing a rootkit on every PC that their CDs touch! Brilliant.
I've worked on some projects that used XML, but instead of using a parser to generate it correctly (which is non trivial due to nesting, encoding, and escaping issues) and using a validator on the output, the code just spewed text that resembled XML on a good day, and broke client code on a bad day. Basically my rule is, if you want to use XML in an app, you need to at least validate the output regularly, such as in a test suite that doesn't get deployed with the app. Fitting a parser into the app is no problem for J2EE apps or anything running on a PC-scale platform, but might be a problem for an app running on a phone or similar limited-resource platform. In that case maybe XML isn't a good choice anyway. XML solves a lot of problems, and solving those problems takes a big scary pile of specs and a big pile of code that implements lots of little rules. A teeny project that can't fit an XML parser (~650K of native code for libxml 2.4.2's libxml2.so, ~1.2MB for Xerces-J's xercesImpl-2.7.1.jar) probably doesn't have all the requirements that a full XML parser satisfies anyhow. Even if you use hand-generated XML sans parser, you can still run the validator outside of the phone, to make sure that your subset of XML's requirements are being satisfied. In light of all this, it's nice to see that someone far more clued in than me about XML has published HOWTO Avoid Being Called a Bozo When Producing XML. Namespaces confuse me sometimes. Unicode confuses me most of the time. External entities confuse me most of the time. I kind of hate all this crud that you have to learn to use XML, but I hate even more the bugs that pop up if you just make up your own format and have no way to check it other than writing a huge program to check it, which nobody does because it takes too long.
For this blog, which is all XML + XSLT using XMLSoft via a command line wrapper (
Point is: use a parser, use a lightweight quasi-XML parser, or roll your own totally non-XML language that's easy to parse and generate. Just do something other than spewing strings and assuming that you didn't make any mistakes anywhere.
I found a really cool example of somebody using CSS 3 features to do something I'd never seen before but which is really cool. I was looking at part of the Subversion FAQ. Note the red box around the particular targeted part of the FAQ. Cool! There's a :target selector now; you can see it used in http://subversion.tigris.org/branding/css/inst.css where it's used like this:
Happy New Year!
We had a great time last night at the Les Claypool Hatter's Ball at the Fillmore. It wasn't quite as good as last year's show, but it was a great time nonetheless. We got there 45 minutes before the opening act was supposed to start, so that I wouldn't have to stand up the whole time on my gimpy ankle. We luckily found some seats with a decent view upstairs (there are tables and chairs upstairs but downstairs and the rest of the upstairs area are standing only) so that helped a lot. It was especially nice to be able to look down at the folks downstairs so we could see all of the freaky hats that people wore. We were hatted too: Kim had a blue hat that reminds me of Blueberry Muffin from the Strawberry Shortcake cartoon and toys, and mine was a leopard fez. We didn't stand a chance against the squid hat or the giant eyeball (a reference to The Big Eyeball in the Sky). Thus we didn't win the fabulous prizes (which Les promised would be not just fabulous but orgasmic), which turned out to be a George Foreman grill, a deep fryer, and a microwave oven.
If you're writing a startup business plan, or trying to get anything done, here's a not to-do list. Some of the items are a bit too accurate...
I mentioned a month ago that I wasn't really spending any time playing video games lately, which is still true. I started on Undying but as usual it got pretty boring pretty quickly. There are some spooky bits, but basically it's yet another first person shooter where sometimes you get a new gun, sometimes you have to kill something that doesn't just die when you shoot it a lot, and sometimes you have to go get a thing from one room in order to open a door. Yawn. It's spooky, yes, but so what? Doom was scary because you were always in danger of coming around a corner and having a monster jump out and go "boo!" too. As usual the world you're in is a vast, sprawling complex of nearly empty rooms containing only weapons and stuff to open other rooms with, and the occasional bed or chair. As usual the geometry of the game map has nothing to do with a real house, so it's shaped more like a space station than a box that's internally divided up into rooms with walls. As usual the gameplay is totally linear. You can't go back to where you were before. You can't explore the house except for a few empty rooms and the place you're supposed to go to advance the story. There are no decisions to make whatsoever except what gun to use at any given moment. The story universe may be interesting, but the game design itself is a yawner. Bad guy coming at you? Shoot it. Door locked? Try the next one. Thing in the room? Pick it up. Person in front of you? Talk to them. Don't bother paying attention, because nothing they say matters. You can't make any decisions so no information they give you is of any use at all. That's basically the whole game.
But every now and then I think back to the games that I did like. Tekken was fun because it was an immediate sort of thing that you could play against someone else, and be done with in a few minutes. Gran Turismo 3 was fun because it was pretty realistic, but I did get tired of the tedium of having to actually drive like a race car driver for the duration of a real race. I gave up on that pretty quickly. It's like a Cessna flight simulator that makes you earn a pilot's license in real time. Why bother?
But I did like the part of the game that let you race really expensive cars, particularly my longtime favorite, the NSX. Sadly, NSX production has just ended, and though Honda promises a new supercar to replace it, I found the tame-looking, Civic-like Acura RSX-S to be an unappealing "replacement" for the Integra GS-R that I used to drive, even though reviewers liked its even better performance. So I'm not confident that the NSX replacement will be as exciting aesthetically as the NSX. That kind of doesn't matter, because the NSX's that are out there are slowly aging their way into my price range, should I ever need to own a car again, and should that need not include anything resembling cargo room. In the meantime I can wonder what it would be like, and GT3 helped me do that. Or so I thought. No, a simulator on a console really doesn't tell you much about what the real feeling of racing a car is. This was learned by BBC's TopGear host Jeremy Clarkson in his comparison of an NSX on PS2 vs real life. Yeah, in real life it's apparently much harder because it's your actual ass on the line and those sickening sensations of weightlessness and the suspicion that just a bit more speed might result in a crash make you take it a bit easier than in a video game. But it still looks like a hell of a lot of fun. And it only costs as much as 160 PlayStation 2's! (Plus gas and maintenance...)