Showing posts with label technology. Show all posts
Showing posts with label technology. Show all posts

Friday, September 12, 2008

Google's Search UI

I wanted to make a note of a couple more Google blog posts that point at some more cool technology. No, I'm not talking about Chrome or the Chrome comic book. Rather, I'm referring to two recent posts by Ben Gomes. In the first post, Ben talks about the principles that guide the design and evolution of the Google search user interface. Of course, any talk of the evolution of the search interface raises a natural question:

A common reaction from friends when I say that I now work on Google's search user interface is "What do you do? It never changes." Then they look at me suspiciously and tell me not to mess with a good thing. Google is fine just the way it is -- a plain, fast, simple web page. That's great, but how hard can that be?"

Turns out it's not as easy as it may seem. Ben discusses a number of principles that guide UI development and provides several examples of the principles in action. One of the most interesting points he makes is related to the goal of search:

...to get you to the web pages you want as quickly as possible. ... This goal may seem obvious, but it makes a search engine radically different from most other sites on the web, which measure their success by how long their users stay. We measure our web search success partly by how quickly you leave (happily, we hope!).

A key part of UI development is experimentation on live traffic. In his second post, Ben describes a series of UI experiments ranging from big, prominent changes to tiny, subtle ones.

... we test almost everything, even things that you would think are so small that we could not possibly care (nor could they possibly matter). In fact, small changes do matter, and we do care.

Check out these posts for a glimpse of the care with which the Google search UI is designed.

Friday, July 18, 2008

Ranking Technology at Google

In a previous post I mentioned that Amit Singhal had promised to have a follow-up post focusing on the technology underlying Google's search algorithms. Earlier this week Amit delivered on his promise with a fine post entitled Technologies behind Google ranking. He says that the driving force underlying ranking is to give users what they want:

Search in the last decade has moved from give me what I said to give me what I want. User expectations from search have rightly increased. We work hard to fulfill the expectations of each and every user, and to do that we need to better understand the pages, the queries, and our users.

Check out the post for more detail.

Thursday, July 10, 2008

Technology at Google

There's been a recent flurry of interesting blog posts describing different pieces of technology at Google. Udi Manber introduced the work of the Search Quality group at Google (I work in this group). His post is the first in a series of more in-depth posts. Here's my favorite bit in Udi's post:

...but the goal is always the same: improve the user experience. This is not the main goal, it is the only goal.

Amit Singhal followed up on Udi's post with a discussion of the philosophy underlying Google's ranking algorithm (Amit has promised a follow-up focused on technology). Here's how Amit describes our philosophy:
1) Best locally relevant results served globally.
2) Keep it simple.
3) No manual intervention.
In a separate series of posts on how data is used within Search Quality, Paul Haahr and Steve Baker write about using data to build language models. They observe that:

By analyzing how people use language, we build models that enable us to interpret searches better, offer spelling corrections, understand when alternative forms of words are needed, offer language translation, and even suggest when searching in another language is appropriate.

Matt Cutts follows up with a post on using data to fight web spam. He says:

Our logs data helps ensure that Google detects and has a chance to counteract new spam trends before it lowers the quality of your search experience.

Outside of Search Quality, I'm particularly pleased about the recent announcement to open source protocol buffers---our data interchange format. Protocol buffers are pervasive inside Google and are a very effective way of encoding "...almost any sort of structured information which needs to be passed across the network or stored on disk. "

Two other announcements are also worth highlighting: Google's C++ testing framework and Google's C++ style guide have both been open sourced.

Finally, here's a video of the Google Factory Tour of Search held back in May.



I have a short segment in there (at about the 65 minute mark) describing some of the work our group has done on query understanding. Earlier in the video (at about the 56 minute mark) Trystan Upstill talks about our group's work on International Search Quality (which is Amit's first point---best locally relevant results served globally). And earlier still (at about the 46 minute mark) Johanna Wright talks about Universal Search.

Tuesday, May 27, 2008

Remote Agent

The Phoenix Mars Lander touched down successfully on the surface of Mars on Sunday. It was no mean feat. The landing sequence was considered so risky that the official mission page on the NASA web site describes it as Seven Minutes of Terror! They note that

...in the international history of the space age, only five of 13 attempts to land on Mars have succeeded.

Check out this great picture of the part of the landing sequence where Phoenix descends into the Martian atmosphere with its parachute.

A crucial mission sequence is called a critical sequence. In a critical sequence you get one and only one chance to to execute the sequence right, with a failed sequence implying a failed mission. Critical sequences include landings (like the one that Phoenix executed flawlessly), orbit insertions (such as Cassini inserting itself into Saturn's orbit), and some flybys (such as Stardust's flyby of Comet Wild 2 to collect and return comet dust).

Talk of critical sequences reminds me of our work on the Remote Agent. I was at NASA through much of the 1990s working as a research scientist in an Artificial Intelligence (AI) research lab. Some time in 1995 the NASA Administrator, Dan Goldin, gave an inspiring speech. In it he outlined a vision that would allow NASA to significantly step up space exploration in spite of shrinking budgets. The key, he said, was to develop new technologies in such diverse areas as propulsion, MEMS, and autonomy. And he established the New Millennium Program to carry out his vision.

We (i.e., a group of AI researchers) were energized by this speech and decided to do something about it. Specifically, we thought that we could build a Remote Agent---an on-board autonomous agent to provide high-level control to achieve mission objectives. We pitched our idea to spacecraft engineers at JPL. Naturally, the spacecraft engineers were skeptical---after all, what did a bunch of AI researchers know about spacecraft autonomy? But they were open minded---they said they'd believe us if we could show them how the Remote Agent would handle Cassini's critical sequence (Saturn orbit insertion) in a simulated environment. And to make sure that the simulated environment didn't abstract away the hard parts of the critical sequence, they assigned an accomplished spacecraft engineer to lead the project.

We worked hard for the next 5 months or so, leading up to a demonstration of the system to various senior engineers at JPL. Everything was going smoothly, when suddenly an unexpected bug hit us (unexpected in the sense that it was not part of the demonstration script, though we were aware of this bug in our earlier testing but hadn't had the chance to track it down and fix it). The bug was in the communication between the Remote Agent and the underlying spacecraft simulation, and it manifested itself as a failure of the main rocket engines just as they were being turned on to slow down the spacecraft in preparation for orbit insertion!

The Remote Agent responded beautifully to this unexpected turn of events! It noticed that the main rocket engine had failed, switched to the back up engine, replanned the insertion sequence, and successfully inserted the spacecraft into orbit around Saturn (all in simulation of course---the real Cassini mission didn't use the Remote Agent!).

The success of this demonstration led directly to doing this for real on Deep Space 1, the first of the New Millennium missions. On May 17, 1999, the Remote Agent took control of Deep Space 1 to start the first of two on-board autonomy experiments. Being in mission control during these experiments was one of the most thrilling moments of my professional life!

You can read all about the Remote Agent here and about the actual experiments here. No discussion of a space mission is complete without a photograph of the spacecraft. I wanted to include a photograph of me with Deep Space 1 (in the testing bay at JPL). Unfortunately, I couldn't locate that photograph :-( So instead, you'll have to make do with the following poster. I have a big version of this poster at home---a parting gift from my NASA colleagues when I left NASA.


Sunday, May 25, 2008

Phoenix Mars Lander

Looks like the Phoenix has landed! I am always amazed by how complex space missions succeed when the engineers get essentially only one chance to get it right.

Check out the first images from Mars here.

Friday, May 2, 2008

Securing your wireless network

In a previous post I questioned the utility of securing your wireless network. Nonetheless, suppose you do want to secure it. Ars Technica has this great post on the ABCs of securing your wireless network. Check it out if you're at all concerned about the issue. Here's the bottom line:

Generally speaking, any router that supports WPA is "good enough" in terms of its overall security. WEP, as we've previously stated, is an "only if you must" protocol, but it's still a better option than transmitting in the clear.

WEP (Wired Equivalent Privacy) is an older security standard than WPA (Wi-Fi Protected Access). While there are different flavors of WPA available, Ars Technica notes that the differences between them are small compared to the improvement of WPA over WEP. So use WPA (unless you want your Nintendo DS to connect to your network--the DS doesn't support WPA).

Early in the article they debunk various myths that:

...do more harm than good by lulling the end-user into a false sense of security.

These include hiding or changing the SSID (the name of your wireless network), disabling DHCP, or filtering MAC addresses. This last method happens to be the one I use currently. From a security perspective:

The problem with filtering by MAC address, however, is that these addresses are easily faked and readily detected by anyone using appropriate monitoring software.

However, it may be appropriate given my earlier post about not really needing security:

...filtering MAC addresses is the only one with even a minimal level of value. MAC address filtering can keep obnoxious and non-tech-savvy neighbors from easily freeloading on your wireless network, but it won't do much else.

Wednesday, January 16, 2008

Wireless network security

Bruce Schneier, a well known expert on security and applied cryptography, writes an interesting blog called Schneier on Security. Recently he wrote a post on his open wireless network. That's very interesting! Here's a security guru and he's arguing that you should not secure your wireless network (even though he agrees that the WPA protocol is very good):

To me, it's basic politeness. Providing internet access to guests is kind of like providing heat and electricity, or a hot cup of tea. But to some observers, it's both wrong and dangerous.

Schneier is unmoved by concerns that strangers might use his open wireless network to indulge in various criminal, or at least unseemly, activities. More interestingly (to me at least):

I'm also unmoved by those who say I'm putting my own data at risk, because hackers might park in front of my house, log on to my open network and eavesdrop on my internet traffic or break into my computers.

And why is he unmoved? It's because:

...my computers are much more at risk when I use them on wireless networks in airports, coffee shops and other public places. If I configure my computer to be secure regardless of the network it's on, then it simply doesn't matter.

And this is essentially the policy I've followed---almost. I dont' encrypt by wireless network. Instead, I depend on the firewall running on our Macs to protect us from prying eyes (if there are any). And I'm assuming that any eavesdroppers on our Internet traffic won't see anything sensitive---if it's sensitive then it had better be going over https, for otherwise there are plenty of other points on the Internet where plaintext passwords and such can be intercepted.

However, I have enabled Mac address filtering, but that doesn't really protect you from a determined hacker. I guess I don't quite agree with Schneier's "basic politeness" argument above.

Talking about open wireless networks, when I was in India in December I found an open wireless network at my parents' house (my parents don't have Internet access). So I used it to quickly read my email. But later my sister told me that in India Internet access is paid for by the megabyte downloaded...! So, unlike in the US, my use of the open wireless network really did amount to "stealing" from my parents' neighbor...:-( After that, I wasn't too keen on using the open network.

Friday, January 4, 2008

Backup and recovery

Some time ago I wrote about backing up ones home computer. Since that post, I started down the path of developing a backup system: I bought an external hard drive. On our older computer, I even acquired some shareware backup software (Deja Vu for the Mac) and was backing up my home directory regularly. However, we then bought a new computer (a beautiful new iMac with a 24" monitor), and I stopped doing regular backups (my version of Deja Vu didn't run on the new Mac and I didn't upgrade).

That's when disaster struck... I was traveling for much of the last 3 weeks, and while I was gone the disk drive on the new Mac failed completely. AppleCare couldn't restore any data from the drive.

Fortunately, by a stroke of pure luck, I had backed up our main home directory (the one with our photographs) to the external drive the night before I left on the trip! Nothing fancy---I simply used the Finder's drag and drop functionality. So I was able to restore most of our data. Not all of it though: the morning of my trip I realized that my camera still had some photographs, so I downloaded them onto the computer and deleted them from the camera. Those photos were not backed up and we've lost them... :-( We also lost some documents created after my trip began, but nothing of much importance.

AppleCare put in a new drive for me, and I've been able to get the system back to the pre-failure state. But now I'm looking at a better backup system. Here's what I have in mind:
  • I'm probably going to use SuperDuper for backup. It's shareware that comes highly recommended.
  • I'm going to try and configure SuperDuper to do automatic backups. That way our data's safety won't depend on dumb luck!
  • I'm going to buy a portable drive, try and do a regular (weekly?) manual backup to it, and store it at work. That will protect against loss due to burglary.
Hope all this works. Do you have a reliable backup system? If not, you better do something. Apparently, disk drives do fail!

Sunday, June 17, 2007

Ed Lazowska's defense of Computer Science research

Scott Aaronson writes the Shtetl-Optimized blog. Scott is a post-doctoral researcher in computational complexity and quantum computing, and a soon-to-be Assistant Professor of Computer Science at at MIT. I don't know Scott at all, but I enjoy reading his blog. His recent blog post is a report from FCRC (Federated Computing Research Conference). The most interesting part of this post was his summary of Ed Lazowska's plenary talk on "Computer Science: Past, Present, and Future". Lazowska is at the University of Washington (where he used to be the Chair), and as Aaronson puts it: this is the guy we want in charge of our field. Lazowska's talk was a rousing defense of Computer Science research.

I was able to locate the slides of Lazowska's talk. I won't summarize the talk; for that just read Aaronson's post for the highlights. But I must draw attention to the next to last slide entitled "Dispel these Myths". In that slide Lazowska highlights a whole series of myths about Computer Science research including:
  • Programming is a solitary activity
  • Eventually, all the programming jobs will be overseas
  • Computer science lacks opportunities to make a positive impact on society
  • Computer Science lacks compelling research visions
  • ... and many more
Presumably in the talk itself, he proceeds to dispel all these myths. The slides were good, but I wish I could have seen the talk.

Which brings me to my final point. My friend Alon (who was, until recently, a Professor of Computer Science at the University of Washington with Lazowska) had a recent blog post on SigTube. The idea is to have a short 5 minute video presentation to accompany each paper presented at a conference. The idea is that:

A 5-minute presentation (done well) can give quite a bit of information and insight about a publication, certainly more than the 100-word abstract or the paper's introduction.

I think this is a great idea. The only difficulty might be the logistics of getting all these video presentations done. If this is too hard to do, I hope conferences will at least do the following: videotape all the plenary sessions and publish them on some video sharing site. These talks are often excellent overviews of a field, where it's been and where it's going. I know I'd love to watch such talks. And I would start by watching Lazowska's talk!

Friday, May 25, 2007

Refocus your images after the fact

Have you ever taken an out-of-focus picture and wished you could correct your mistake? Or has your artistic side desired to blur parts of a picture even though everything is is sharp focus? After all, you can use Photoshop to make all sorts of modifications to your photographs. But even Photoshop can't help you: you're stuck with the depth of field you used when you took your picture.

That may be changing soon. Ren Ng of Stanford University was awarded the 2006 ACM Doctoral Dissertation Award for his work on Digital Light Field Photography. Ren and his colleagues at the Stanford Graphics lab built a digital plenoptic camera, described in the tech report "Light Field Photography with a Hand Held Plenoptic Camera". A plenoptic camera is

...a camera that samples the 4D light field on its sensor in a single photographic exposure. This is achieved by inserting a microlens array between the sensor and main lens, creating a plenoptic camera.

This microlens array is analogous to an insect's compound eye.

Each microlens measures not just the total amount of light deposited at that location, but how much light arrives along each ray. By re-sorting the measured rays of light to where they would have terminated in slightly different, synthetic cameras, we can compute sharp photographs focused at different depths. ... we demonstrate that we can also compute synthetic photographs from a range of different viewpoints.

And that allows one to refocus the image even after one has taken the picture! You can see examples of refocused images if you follow the above link to the tech report. Utterly fascinating stuff! Ren is now at a startup called Refocus Imaging that is commercializing these ideas.

Sunday, April 29, 2007

Backing up your home computer

Do you backup your home computer? How often? Chances are that you often think about backing it up. You might even have actually set up a "system" to back it up. And better still, you might even have backed it up a few times. But it's now been a few months since you last did a backup...:-(

That, at least, describes my situation. I think often about backing up our home computer. My "system" is to backup to CDs and DVDs, so it's quite a slow, cumbersome process. So I don't do it often---my last backup is dated 8/5/06...:-( Now that's called living on the edge.

I've been wondering about the right backup strategy. Something that will just work once it's set up.

One option is to buy a large external hard drive and set up nightly/weekly automatic backups. That will certainly create duplicate copies of your precious photographs, videos, music, and other documents, and will protect you from hard drive failures. But what if there's a house fire that destroys your computer and hard drive? (As we know from our experience, this isn't such an unlikely event...!) Or what if your house gets burgled? Last year my office mate had her house burgled and the thieves focused on taking all and only the electronics in the house---they failed to take an expensive pair of skis that were lying around, but did take all the computers and hard drives they could find! (Like our house fire story, this story also ended well---the police caught the thieves and almost all the electronics were returned safely.) So I suppose one can build extra redundancy into the system to make multiple hard drive copies and keep one of the hard drives in a safe(r) place. Sriram has adopted this strategy.

Another option is to use various online backup services (Mozy, Carbonite, etc.). This does seem like an attractive option, but has some problems for me. I would need a Mac backup solution, and most are PC-only. Also, we currently run Mac OS 10.3 (Panther), while most solutions expect you to be running Mac OS 10.4 (Tiger). So this might be yet another reason to buy a new computer (upgrading this computer to Tiger seems sub-optimal). And then there's a question of how reliable these companies are. Are they going to be around for the long run? A solution backed by one of the major Internet companies might be safer. Are you comfortable storing sensitive material there (such as documents related to your finances)? Frankly, I don't think I am---I'd rather back those up to CDs/DVDs and store those safely.

What do you do?

Friday, April 20, 2007

Aptitude

Yesterday my officemate, David, and I were upgrading the Linux on our desktop machines from Red Hat to Ubuntu. Unfortunately, the upgrade didn't go smoothly at all. In one failed attempt, the machine sat there forlornly with the following message:



We were wondering if this was a not so subtle commentary on our lack of it, or whether we had lost it all and they were having trouble finding it...:-)

Thursday, April 19, 2007

Online pedometer

Do you take long walks or go running in your neighborhood? Or do you take your kids for bike rides? Have you wondered how far you went on each of these trips? Of course you did! And you probably had a pretty good estimate based on some idea of how fast you were going. Maybe you know you run between 9 and 10 mins/mile (unless you're Sub, in which case you run at 7 mins/mile!). Or maybe you know that you walk about 3.5 miles/hour (unless you're Puru walking a marathon and going at a zippy 3.83 miles/hour!). But even after all your estimating, surely you're left with this niggling question: how much did you actually go?

Well, don't let that question niggle you any more: you need GMaps Pedometer! This is a great site, built on top of Google Maps, that allows you to lay out your route on a map. It tells you how long the route is, how long each leg of the route is, how many calories you burnt (I haven't used this, but it sounds interesting---isn't that the whole point of being out and about in the neighborhood...:-), elevation, ... It's really cool, and it takes out all the guesswork from your estimates. You can now veg out in front of the TV (or in front of your computer reading blogs) serene in the knowledge that you just finished a 4.53 mile run!
/* Google Analytics tracking */