It's as easy as pressing a key when the language is being spoken—let's say you're trying to order some food in Spanish. It translates what you say into the phone on the fly, and then reads the Spanish version out loud, so the waiter can understand. They can then respond in Spanish, and the phone will convert it to English, speaking it out loud.
You can imagine how crucial this would be to travellers—and while it can only compute English and Spanish for now, I'm betting Google's working on all manner of other languages. Check out a preview video of the app above, from a few months ago, or a user video below.
Android users with 2.1 or higher can download the new Google Translate now, which has a few other improvements built-in as well. [Google Blog]