Oxyfish

Cloud computing means lots of different things, and much of it is hype. At Yieldex, we’ve been using cloud computing, specifically Amazon Web Services, as a key part of our infrastructure for the better part of a year, and we thought we’d pass on a few of our lessons learned. As you might expect, the services we use have trade-offs. If your challenge fits within the parameters, cloud computing can be a huge win, but it’s not the answer for everything.

All of these lessons are the result of the hard work of our entire engineering team, most notably Craig and Calvin. These guys are among the best in the world at scaling to solve enormous data and computation problems with a cloud infrastructure. We could not have built this company and these solutions without them.

For a startup, there are a number of compelling reasons to use a cloud infrastructure for virtually every new project. You don’t get locked into a long-term investment in hardware and data centers, it’s easy to experiment, and easy to change your mind and try a different approach. You don’t have to spend precious capital on servers and storage, wait days or weeks for them to arrive, and then spend a day or two setting them up. If your application scales horizontally, then you can scale additional customers, storage, and processing with minimal cost and time delay. All these things are touted by cloud providers, and basically boil down to: focus on your business, not your infrastructure.

Sometimes, however, you do need to focus on the infrastructure. We provide our customers with analytics and optimization based on our unique and proprietary DynamicIQ engine. Our first customer was a decent sized web property, and we were able to complete our DynamicIQ daily processing on several gigabytes of data using just one instance in less than an hour. Our next customer, however, was 10x the size. And the one after that, 10x more – hundreds of gigabytes per day. Fortunately, we had designed our DynamicIQ engine to easily parallelize across multiple instances. We spent some time learning how to start up instances, distribute jobs to them, and shut them back down again, but because we had designed the engine for this eventuality, we were able to use the cloud to cost-effectively scale to even the largest sites on the web.

We also have BusinessIQ, which is basically an application server that provides query processing and a user interface into our analytics. Initially we started with this server in the cloud too, but as we bumped up against other scalability issues, we found that the cloud doesn’t solve every problem. For example, we provide a sophisticated scenario analysis capability. To calculate a “what-if” scenario requires processing a huge amount of data in a very short time. For our larger customers, a single cloud instance did not have enough memory to perform this operation. Trying to stay true to the cloud paradigm, we implemented a distributed cache across multiple instances, but this didn’t work well because of limitations on I/O. We ended up having to go to a hybrid model, where we bought and hosted our own servers with large memory footprints, so we could provide this functionality.

We have been very happy users of the Amazon Web Services cloud, and not just because we won the award. We would not have been able to get our business of the ground with out the cost effective scalability of the Amazon infrastructure. While it’s not for every application, for the right application, it truly changes the game.

We won! Out of nearly 1000 startups who applied, we won!

This is a great validation of our fantastic technical team. We have been chosen as one of the most innovative users of Amazon’s cloud computing technology. We could not have done this without your hard work. Thank you!

This was a great experience for us. The Amazon team was very professional throughout, the event was well-managed, and they even made a cool video featuring our development team. The press release went out tonight, and there was even a blog post that beat mine.

Here’s the blow-by-blow, for those who want all the details:

We pitched the panel of judges, all senior execs of Amazon, at 1pm. They had 50min presentations from each of the seven finalists, and had been going since 7am. We did our standard pitch, and did a great job talking about how important AWS is to us. They seemed to appreciate the presentation, but were somewhat poker faced, so while we felt we did a good job, it was hard to tell their reaction.

Later in the afternoon, we had a “VC speed-dating” event, where we had 10 minutes with each of 5 VCs. The firms were all first-rate (BlueRun, Hummer Winblad, Madrona, Greylock, and CMEA). Our product is pretty complex, so it’s hard to get across in 10 minutes, but we did our best, and each of the VCs seemed to get it quickly enough. All were interested in following up, but again, hard to tell how we ranked.

Then there was a reception while the judges and the VCs deliberated. They had invited 200 other startup people to come hear how the seven finalists were using AWS. My guess is closer to a hundred people were in the room, and we had to do another 10 minute presentation on AWS, with slides, to this group. We managed to do it in only 5-6 minutes and get our message across. Finally, around 9pm, it was time to announce the winner. We were jubilant when they picked us – I let out a shout of joy and a fist pump, to the delight of the audience.

Andy Jassy, the SVP of AWS said some nice words and gave us the traditional golden hammer. We were then invited to take a whack at an old rackmount server they had, to symbolize the destruction of our own servers. John and I both hammered it pretty hard, but we barely dented it – those steel frames are tough.

Then everyone came up to congratulate us, and we shook hands with big grins on our faces. We took a couple of pictures, approved the quote in the press release, and talked with the Amazon folks some more. All great stuff.

Finally, we headed out for a celebratory dinner. Once again, thanks to everyone in the company for your hard work – we did the talking, but we could not have done this without all of you.

Hooray!

Once in a while I feel the need to write some actual code. I’m really a programmer at heart, and I find it incredibly satisfying to think through a problem and create a useful solution. For me it’s more interesting than crossword puzzles, and the end product is (sometimes) more valuable.

For a variety of obvious reasons, it’s not a good idea for me to get my fingers into Yieldex production code, so I end up scratching this particular itch with small side projects. This one took me a couple of hours, and hopefully will be useful for enough people for it to have been worth it. In any case, I learned a lot, so it was worth it for me.

I’ve written about CellarTracker before. I’ve been a user for years; I love the site, and I love the business. But, I find some of the UI to be less than perfect. Here’s an example: when I get a wine offering in the mail, there are usually a number of different vineyard designations, and I get an allocation of a few bottles from each. I haven’t found a way in CellarTracker to enter 2 bottles each from 10 different designations without doing an incredible amount of clicking around and waiting for pages to refresh. So, I set out to solve this problem with a “bulk purchase” mechanism.

There are a number of different ways to tackle this problem, so the first thing was to decide on an approach. I’m still a bit 1999 when it comes to the coding for the web, so my first idea was to set up a server-side program. Then I started thinking about the complexity of screen-scraping and parsing, and robustness in the face of potential CellarTracker updates. Then there are the security issues of passing usernames and passwords so my server could log in to CellarTracker. Finally, I realized I didn’t really want to be responsible for keeping the service up and running, so I almost bailed on the entire idea.

Then I remembered GreaseMonkey for Firefox. Cool – an opportunity to enter the 2000’s in web programming, and polish up some JavaScript skills. And, it got around all of the above problems in a neat client-side way. The only real issue is that it works only with Firefox, and for most people would require an install of GreaseMonkey and the script itself.

I started by installing GreaseMonkey and a couple of web development tools, notably the DOM Inspector, the Javascript Shell bookmarklet, and then later the Web Developer Toolbar. I read quickly through the Dive Into GreaseMonkey book, and then just started coding. I was pretty excited that in only a few minutes I could build a script to automatically change a CellarTracker page upon load.

After a couple hours of experimentation, some heavy shell use, and a bit of DOM inspection, I had something up and running. I created a test account on CellarTracker, and entered a bunch of purchases. Success! I finished up by spending a few minutes on data validation, trying to make it obvious when something went wrong and how to fix it.

The best part about this project for me was learning one level deeper about how Javascript and the DOM work. Javascript is a much more powerful language than I remembered from 1999, and I have much better understanding of how Ajax and many modern web sites work. And it was fun.

The final step was posting it in the new/old OSS directory here on Oxyfish, and writing this entry. I also posted a note in the CellarTracker forum, in case anybody else wants to use it. I’m very happy with how it turned out, and am looking forward to the next wine offering in the mail, so I can start saving time.

Install the CellarTracker Bulk Purchase extension (requires Firefox and GreaseMonkey)

I use the Wordpress Database Backup plugin to send automated backups of my blog to a dedicated Gmail account on a daily basis. When the disk filled up on my server, the blog database was corrupted. Fortunately, it was the work of about 10 minutes to restore from the backup. Thank heaven for fire-and-forget backup strategies!

When I rebuilt my old machine about three years ago, I had limited time to get things working, so I focused on the essentials (main web site, email, and blog), and ignored the rest. One thing that fell through the cracks was a site I had been running since 1998, guid.org. From the site:

GUID.org is an Internet service that assigns anonymous random user IDs to web browsers. These anonymous IDs can then be used by other web sites for many purposes. For example, a site may use your GUID to recognize you when you return. GUID.org does not collect or store any information about users – see our privacy policy.

GUID.org was conceived back in 1998 when it was still new technology to insert a “web bug” to correlate users across domains. Now that technology is old hat, but I still think there may be a use for a universal GUID that can be shared by lots of sites.

If anyone comes up with a really great plan for how to use this technology (and domain) in this modern world of internet advertising, I’m all ears. I’m sure there’s a pony here somewhere…

I finally decided that I didn’t need two blogs, so I merged them into one. Blogs do seem to naturally divide into several types, one being the thoughtful occasional post blog, and another being the short blurb with a link type. So, I originally decided to have two different blogs, Tom’s Tech Toys for short links, and Oxyfish for longer items. But managing two blogs is just silly, especially since I don’t post that much anyway. So I merged the other one into this one. Read on if you want technical details.

Read the rest of this entry »

Sorry for the lack of posting. You’ll notice a few posts that have stacked up over the last few weeks. I had a few technical difficulties that I had to recover from, and with a full-time job and 3 kids at home, I don’t have a lot of time for diagnosing tech problems.

I recently had decided to play with Ruby and Rails, so I upgraded a few packages on my FreeBSD system to be able to deploy some toy applications I was writing as a learning exercise. Unfortunately, at some point along the way I broke something, and Wordpress started giving me “Fatal error: Call to undefined function: xml_parser_create()”. I tried using Flock to post, and it gave me the same problem, although I had to turn on error logging and look at “blog.log” to see it.

I started doing some digging, and found that xml_parser_create() is part of the XML extension to PHP, so I tried rebuilding that – no dice. Then I updated all my ports, and tried again. Now it won’t even compile. The error is:

libtool: ltconfig version `' does not match ltmain.sh version `1.3.5-freebsd-ports'

Hmm. Google didn’t help on this one. I tried rebuilding PHP 4.4.6, but that failed as well. Googling that problem again didn’t help. Hmm.

After still more digging, I figured out that I’m running FreeBSD 4.1, which has recently been End-of-Lifed, so the ports are no longer guaranteed to work. Arrgh! Now somewhere I’m going to have to find time to upgrade to FreeBSD 6.

I still needed to fix the problem, so I tried using the packages system and installing some old versions of the packages. That totally screwed up everything, so I quickly backed that out.

I went back to the beginning and turned on PHP error logging. When I restarted Apache, I got this in the error log:

PHP Warning: Unknown(): Unable to load dynamic library '/usr/local/lib/php/20020429/xml.so' - Shared object "libexpat.so.5" not found in Unknown on line 0

Now I’m getting somewhere – the library is just not found. I found libexpat.so.6, but not 5. After rebuilding libexpat, I found that the version 2 I have, which I must have upgraded to as part of the Ruby/Rails install, is the .6 library version. I couldn’t easily find the .5 version, but Google pointed me to a great idea. Just symlink the .6 version to .5 to fool PHP. Bingo! It works!

System administration can be a pain – I admire those who do it for a living. I do it once in a while, just so I remember how painful it is sometimes. Now, to post a few entries that have stacked up.

[I started writing this at the Microsoft VC Summit a few weeks ago, but tax day spurred me to complete it.]

Other than hardcore libertarians, I think there are few who would debate that government funding of fundmental research is a good thing. Much of this research is done at top universities funded by government grantd, but there are also institutions like DARPA, NASA, and NIH that are directly funded. Most corporations, with their focus on quarterly earnings, have too short a timeline to spend significant amounts of money on research that doesn’t have an obvious return on investment in a relatively short time frame.

There have always been a few exceptions, and what is interesting is what they seem to have in common. For example, Bell Labs springs to mind as a great exception. They produced literally thousands of innovations, most of which were (at the time) commercially unusable. Another classic example is Xerox PARC. Once again, tremendous innovative and fundamental research, with little commercial application. What is interesting about both these companies is they were essentially monopolies, and highly profitable, such that their products were referred to as “taxes”. Today we see companies like Microsoft and Google engaged in similar research efforts (although Google’s is pretty young still).

I heard Steve Ballmer speak the other day, and he boasted several times about Microsoft earning $20 billion last year. Many who were with me groaned about the egregious “Microsoft tax” and expounded on how much better the industry would be if everyone used Linux and OpenOffice and the $20 billion were returned to the users.

This prompted a spirited discussion at lunch, which included some long-time Microsoft execs. The “Microsoft tax” is pretty small for each individual. Which creates the greater good: giving each computer user a small amount of money back to spend as they wish, or allowing Microsoft to engage in fundamental research that may improve the lives of everyone? Viewed this way, it looks much the same as increasing the income tax 0.01% to pay for NASA. Of course, this only applies to the $1 billion or so that Microsoft spends on research. The other $19 billion that is dividended back to shareholders is more like a reverse Robin Hood – take from computer users, give to MSFT stockholders.

Now, there are plenty of egregiously profitable companies – Exxon Mobil for example – that don’t spend nearly as much on blue-sky research as they could (despite their marketing that says they do). Perhaps instead of legislating lower profits for them, the federal government should consider legislating more pure research? Of course, this would be hard to verify, but it would be a start.

A friend of mine is from New Orleans, and is a huge Saints fan. The weekend they were in the championship game, we were headed to Tahoe with our families, and staying in a house with no TV reception. He threatened to leave early, to get home in time for the game. What to do?

We didn’t even have a broadband internet connection. But we did have a decent cellphone signal, and I have an EVDO phone that I can connect to my laptop. And I have a Slingbox at home. Aha!

Challenge number one: the cell tower wasn’t upgraded to EVDO, so we were stuck with 1xRTT which is much slower. I was seeing about 100Kbps sustained throughput. Fortunately, Slingbox is very good at optimizing available bandwidth, so the picture was okay, although a bit blurry (hard to read the time remaining) and blocky when moving fast. Sound was great, which helps a lot.

Challenge number two: nobody wants to watch the game on the computer, we want to watch on the big screen. Fortunately I have a Mac, which has TV-out. I also carry cables to hook the Mac into the stereo as well as the TV, so we had good sound and good picture.

Challenge number three: my home cable service was inadvertently cut the day we headed up there. So, Slingbox working great, but no TV signal at all. This was stumper. I scoured the internet for other streaming services that could get us the game, but aside from a few questionable-looking sites, I couldn’t find anything.

When all else fails, try asking someone else. I posted a question on LinkedIn Answers. Within an hour, I got two folks from my network offering to let me connect to their Slingboxes. Problem solved!

I connected to a Slingbox that happened to be located in Atlanta. He had a Comcast DVR hooked up, so we could even rewind and fast forward if we wanted, although it was a bit slow to respond. The whole thing worked almost flawlessly, and we forgot at times that we were watching over a pretty slow network connection.

My friend was ecstatic to watch the game, although his ecstasy soon turned to agony. We will be up there again Superbowl weekend, so we’ll have a chance to try it again, although this time it should be our own Slingbox. Now to hook the Slingbox up to the Media Center PC, so I can get my recorded shows…

I did a recent podcast with Steve Bengston about blog marketing. I think it is a good introduction for entrepreneurs starting to think about inexpensive marketing for their startup companies. I talk about how to get started, how to promote your blog and get readership, and the importance of posting regularly and in your own voice. Most likely if you’re already an experienced blogger, there won’t be much new here.

PricewaterhouseCoopers Start Up Show – Blog Marketing

(local mp3)

The origin of CDDB is one of my favorite stories about the evolution of technology, and the discovery of value in places where nobody initially expected them to be. This is how I understand it went: sometime in the 80’s a smart guy gets a shiny new Sun workstation with a CD drive. He figures there must be some way to play his audio CDs on it, so he creates Workman. Then he decides he wants to see the track information while the CD is playing, so he types in the info for his CDs. Of course, he doesn’t want to type it in each time he puts the CD in, so he invents a way to use some of the data on the CD as a key, and creates a database to look it up.

Now, he also gives this program away for free, and as a bonus, includes his own database, just in case there’s any overlap between his CD collection and the next guy’s. Soon other folks are sending their databases back to the original guy to be merged into the distribution, and after a while, it’s pretty big. So, he starts distributing it separately, and other clients start to use it too. Then this internet thing comes along, and another guy has the brilliant idea to make it client-server. A few more clients get written, people start submitting lots of CD information, and the database starts to get pretty huge. Eventually, in a very controversional decision, they decide to make the service a for-profit one, and now according to Wired, Gracenote is “a dominant player in music recognition.”

The story I heard matches reasonably closely with this summary on the MusicBrainz site:

The roots of this project are in a software program called Workman for playing Audio CDs on UNIX systems. Workman had the ability to display the name of the track it was currently playing. An index file was used to store the tracknames for each Audio CD. After a while a large index file with information about thousands of Audio CDs was created by the Internet community.

This was a long time ago. The index file system became more widely used when Windows users started using this index file, but the system was not very mature then. The Windows Audio CD player could use an index file with track information, but the index size was limited to 640KB. This meant that Windows users could not use the large Internet index file without correcting software.

In 1996 things changed when the Internet Compact Disc Database was created. Instead of a flat file with information of thousands of Audio CDs, the client/server model was applied. A single central server called “CDDB.com” could be used to access the information of Audio CDs. This server accepted new submissions of Audio CD information. At that stage the index file was reported to have grown by up to 800 Audio CDs per day. But these numbers say nothing about the quality of the submissions. The number of duplicate Audio CDs that now exist in the database is high — 10 entries of the same Audio CD under a different number is not uncommon. Many entries also contain numerous spelling errors. CDDB.com had no mechanism to correct errors.

Despite this, the system became popular and useful. Things changed dramatically when the open CDDB.com server was bought by a company that wanted to make money from the contributions that users had made. The index file created by the Internet community could no longer be copied. Patents were obtained and granted. A large public outcry resulted, and led to the start of several projects to create an Open Source competitor for the commercial CDDB.com (now Gracenote).

I think that CDDB is probably the first good example of a user-generated database. Now, user-generated content is all the rage, but it is interesting to separate it into things that seem more like databases, like CellarTracker and Wikimapia, versus the more free-form things like blogs. More on this from an investment standpoint in a future post.

I am an inveterate science fiction fan, and I read as much as I can – although these days, with 3 young kids, that’s not much. I don’t think that science fiction is very useful as a good predictor of the actual future, but it helps keep me sharp by thinking about the way things could be. I especially like the short story format, because you can explore one or two interesting ideas without having to do a lot of the plot and character development required in longer forms.

Bruce Sterling is a great writer in the field right now, and I just ran across a recent short story that is linked below. I am also a Simon and Garfunkel fan from way back, so this quote really resonated with me:

The words of the prophet have vanished from the subway walls.

Finally, since I have the above-mentioned 3 small children, I am continually trying to imagine the world they will live in, so I can try my best to prepare them for it. I am always struggling with where to put the boundaries, so my kids can explore the world, but in a reasonably safe and sane way.

The story is a little depressing, but like all good stories, made me look at the way I think about something important to me – in this case, raising my kids – a little bit differently.

I saw the best minds of my generation destroyed by Google

Well, the summer doldrums are in full force, and it’s hard to find the time to blog. I still have lots of ideas, but between vacations, family, and a real job (I can hear you laughing out there), I don’t really have the time to develop any of them.

I recently found an “Idea Book” that Louisa and I used while we were traveling around the world. Inside the front cover it says “Hanoi, Viet Nam, August 2000″. I find it fascinating to peruse, particularly since a number of the ideas have since become technologies in general use. For example, I had the idea of allowing readers to add links to a page, with two variations: actually editing the page (with admin permission) to insert the links, or browser-side with “sticky notes” that only the author can see. The first is basically Wikis, and the second is Mystickies or Diigo.

Since we were traveling at the time, and updating our site Shields Around the World from random internet cafes was a laborious process, the first idea was a travel journal site. Now, of course, there’s TravelPod and TravelJournals and quite a few others, but at the time there weren’t any good ones that we could find. The main idea we had that isn’t quite as well developed yet was the GPS integration, so it’s easy to see on a map where you are. We also wanted to mine emails from travelers for recommendations. I won’t take the time to write up a recommendation of a place just because we visited it. But if a friend emails and asks “how is this place?” I’ll definitely write back with some impressions. If those could be captured automatically, that would be incredibly valuable.

The book lists some fairly generic thoughts, like delivery of ads to PDAs via Bluetooth, and other Bluetooth ideas captured in a previous post, Bluetooth World. I’ll try to list some more ideas in later posts.

My wife has been bugging me for a good family calendar. Our kids (4, 3, and 1) are not necessarily “overscheduled”, but we are an active family, and keeping track of the details and logistics is often challenging. Also, with my wife’s occasional board meetings and volunteer activities, and my own breakfast meetings and dinner events, we had to institute a regular Sunday night manual calendar sync. Sure would be nice to see if my wife has a board meeting before I schedule a dinner with a CEO.

30 boxes looks amazingly cool, but doesn’t sync. Trumba when it was announced seemed like it was going after this problem, but now sounds quite different. Google Calendar is a latecomer, and doesn’t have the features (nor the sync). Yahoo Calendar syncs well, but doesn’t seem to offer multiple calendar integration. So, I never quite found what I needed. A recent PC Magazine article, Manage the Family Calendar, convinced me to look at AirSet. This looks much better, and I’ve signed up. Now I just need to set up the sync for my Outlook and my wife’s Palm Desktop, and we’ll see how well it really works.

When I first heard about Bluetooth in 1999, I thought it was going to change the world. The whole concept of Personal Area Networking made total sense to me, and I was already envisioning a world where various devices about my person could all communicate. I wrote a short essay of what I thought would come to pass in a year or two, but this was before blogs, so I actually hand-wrote it in a notebook.

The thesis was there would be a main “computing unit” that I would keep in my carry bag or possibly (if small enough) in my pocket – this would also include lots of storage. There would also be a “communications unit” that might be the in the same form factor, or might be much more pocketable, like my phone. I would then have peripherals as needed to access the computation, storage, and communications capabilities. The main peripherals I defined were a headset, a watch, an eyeglass display, and a folding keyboard, all communicating by Bluetooth.

The eyeglass display was inspired by a PC Magazine article from years before that showed a prototype that would give a full screen display using a technology like Icuiti’s, and it was “just around the corner” for years. The folding keyboard is fairly obvious, and I actually own one now, although I don’t use it much. The headset has now become ubiquitous, and I see people wearing them at entirely inappropriate times, but that’s a topic for another post. The only one I haven’t seen until today is the watch.

Why a watch, you ask? Well, I wanted to have the exact same opportunity to choose to pay attention to a phone call as I do to choose to pay attention to someone coming in to my presence. This meant that I needed a very discreet and immediate way to recognize that person and make an unconscious decision, the same way I can acknowledge or ignore someone who approaches me when I am busy. Checking my watch discreetly seemed like an obvious way to do it.

Finally, today, I saw that Citizen is introducing a Bluetooth Watch. It is ugly as sin, and I would never wear it, but if there were one that looked professional, I think a lot of people would find it useful.

via Engadget

I like this thing. It sure would be nice to interact with your computer in 3d…

Switched On: Taking control to another dimension – Engadget

I manage my own co-located dedicated server, which runs this site, and about 15 others, including Shields Around the World (more on this in another post). I do this mostly because I’m a geek, but also to keep up with the times, and to have a server for trying out infrastructure technologies on my own. I have hosted with meer.net since they had a tiny back room in Mountain View in 1998. At the time, they seemed to be a couple of ex-Netscape guys who had bought some old networking equipment. During the bust, they acquired tons of great gear for pennies on the dollar, and even picked up a couple of fully built-out hosting facilities for incredibly cheap. They seem to be doing very well, and I’m extremely pleased with the service.

Recently they sent me an apologetic note explaining that they had to assign me a new IP address block, because the one I was using eventually needed to be recovered. They are promising this will never happen again, since this block is one they own. It was something of a PITA, but I got through it in a couple hours. If you aren’t interested in the tech details, skip the rest of this post.

My server runs FreeBSD, and I grew up with SunOS, so it’s pretty similar. But I’ve used enough different Linux machines, etc, that I always have to find where things are. I found the rc.conf where I set up the aliases to listen to the new IP addresses. That’s where I made my big mistake – I changed the primary IP address to the new one, but I didn’t change the default router. Oops! As I later found out, packets could get in, but they couldn’t get out.

Fortunately, meer has a nice virtual console feature. Unfortunately, my SSH key was at work, and once I retrieved it, I found I had forgotten my passphrase. Incredibly, they have the ability to update your public key through their support web site, so after generating a new key and updating it, and waiting a few minutes for the servers to sync, I was on the console. While I was on the support site, I figured out that I had to change the default router, and found a couple of other changes – the DNS hosts, and the NTP hosts – that I had to make.

Of course, when the machine came back up, it found some back blocks on one of the disks, so I had to deal with that. Then I changed the default router, rebooted, and everthing worked again. Whew! It’s a bit nerve-wracking to have your main server down for an hour while you try frantically to fix it.

After that, it was easy to change the DNS in resolv.conf, and update the hosts file. When I checked my server logs, I kept getting “ntpdate: no server suitable for synchronization found” in my logs. I checked cron with “crontab -l” but there was no ntpdate. Finally, by chance, I found an /etc/crontab, which had the offending line. I still don’t know why /etc/crontab doesn’t show up when running crontab -l as root.

The remaining tasks were a little tedious, but easy. Apache needed to be reconfigured to listen to the new IP addresses for the virtual hosts. All the DNS files needed to be updated. For this one, I was a little more conservative. I updated the DNS for oxyfish.com and one other domain, but have held off on the rest for now.

So, if you had trouble seeing the site for a few days, you know I screwed it up – DNS problems always seems to take days to straighten themselves out. But if you are reading this – you know I fixed it!

Just ran across an incredibly cool password management tool. Basically it uses a “master” password and the domain name of the site you are visiting to create a unique password for each site. The best part: it is calculated (using a modified MD5 hash) in the Javascript – so no web refresh, no transmission of your master password. And it runs entirely from a bookmarklet, with automatic discovery and fill-in of the password field. Or you can use the mobile version to gen a password on the fly. The only minor issue I can see is that if your master password is compromised, then the attacker could get access to all your sites. But that’s not really a problem, because most people use the same password almost everywhere anyway .

Thanks to JohnK for the pointer.

I really enjoyed the Singularity Summit last Saturday. I didn’t go as an official blogger, and I didn’t bring my laptop, so I only have a few basic notes on key ideas that I captured on my blackberry. I went and stood in line with the other “singularians”, a new word I learned that seems to denote far-out folks that think they will live forever amid super-intelligent machines. Much more complete conference coverage can be found at Dan Farber, Down the Avenue, Responsible Nanotech, and Chaos Manor.

I was a little surprised to see Steve Jurvetson on the stage. If I had known, I might have hit him up to get in to the VIP lunch. He did a good job trying to moderate a Q&A session that was a little too wide-ranging with too many panelists.

Ray Kurzweil was first up. I enjoyed his book, and had listened to his speech at Accelerating Change via IT Conversations. This talk was pretty similar, but still stunning in the amount and depth of detail. He was the only person at the conference with facts and figures at his fingertips, and it gave him an aura of credibility that nobody else matched. I would have loved to hear someone argue with his facts and conclusions. Unfortunately the only real dissenters in the group (Douglas Hofstadter and Bill McKibben) just had vague unease with Ray’s conclusions, and no solid numbers or logic to back them up. Douglas was downright petulant, and it was a huge disappointment for me, since I have been in awe of him since I read Goedel, Escher, Bach back in high school. Ray nicely eviscerated Doug’s comments in his wrap-up.

Sebastian Thrun was inspiring and hilarious in his presentation of the features Stanley needed to win the DARPA Grand Challenge. Particularly funny were his video clips of the “autonomous motorcycle” – allegedly from Berkeley – that kept falling over. One interesting question is whether these kinds of approaches are like “climbing trees to get to the moon” – i.e. seemingly in the right direction, but ultimately never going to reach the goal.

Eric Drexler was fabulous, on his usual theme of “atoms are becoming like bits.” He sees a future where manufacturing has the same economies of scale and replication as information technology. I don’t know much about this, but it sure sounds great. I read Engines of Creation a long time ago, but I need to get back up to speed.

Max More was talking about the Proactionary Principle, which I didn’t quite get, but subsequently looked up. He also talked about ensuring that super intelligence equals super wisdom in these intelligences. I submitted a written question during his talk, wondering which, if any, of the human emotions would need to be replicated in such a machine to enable it to effectively communicate with humans, and further if any of those emotions would be required to succeed in a super intelligent, super wise society. I didn’t get picked, though, and I couldn’t stay for the entire Q&A session, so I guess I’ll have to wait until the next one to ask.

Eliezar Yudkowsky had some interesting ideas to share. In particular, the idea that intelligence is the most powerful force we can deploy to improve our world, and that therefore the question for super intelligence gives us the most leverage. Reminds me of a talk by Bill Joy I heard back in 1987 or 1988 about the almost inconceivably powerful Sun 9 (they had just released the Sun 4, we named our first one “speed” because it seemed so fast). He said if you had a program that would take a century to run on the Sun 4, just wait 15 or so years for the Sun 9, and run it in 8 hours.

Bill McKiibben talked a lot about how less is more, and the journey is its own reward, exhorting us to slow down. Of course, when Ray asked him exactly how to do that, he didn’t have any good answers. This talk reminded me of a series of award-winning short stories called Kirinyaga by Mike Resnick, where a group of African tribesmen try to preserve their culture by moving to a different planet. Ultimately, progress wins out, and you only have utopia for a fleeting moment that you rarely appreciate while it is happening. Bill wants to save the environment, but doesn’t seem to get that some of this progress is in advanced energy sources and manufacturing techniques that will be much better for the environment.

I came away pretty impressed with the caliber and the depth of the thinking that has gone into this issue. One of my goals is to find the time and resources to contribute to this conversation.

I had been pooh-poohing the whole idea of podcasts for some time. I downloaded a few, listened to them, and decided they were just amateur radio shows, and not very good ones at that. Who needs that? To be perfectly honest, I generally prefer performing music to listening to it, and in my car I usually listen to KCBS or NPR.

A friend pointed me at IT Conversations. At first, I resisted, because I usually don’t like listening to spoken words, like audio books. This may be because I read much faster than I listen, so I get bored and distracted. I get most of my technical knowledge from reading on the web, and couldn’t imagine really learning anything by listening to talks.
However, I saw a session by one of my favorite thinkers, Ray Kurzweil, so I decided to give it a try. Now I’m completely hooked, and I think there are a couple of reasons. One is that these presentations are designed for a listening audience, so the cadence and the style are very different from an audio book. Of course, they sometimes have slides, which I obviously can’t see, but I still get most of the good stuff. Another reason is I can “attend” conferences without actually traveling anywhere, or missing any days of work. I know I’m missing out on all the valuable networking, but some conferences actually have interesting content, and IT Conversations seems to have most of them.

I have since recommended IT Conversations to a number of friends, and all have found valuable sessions there. A few of my favorites:

• Ray Kurzweil
• Larry Lessig
• Clay Shirky
• Vernor Vinge

Conferences include O’Reilly Emerging Technology, Web 2.0, Pop!Tech and OSCON. Worth checking out.