From Dolphins to Swype: An Inventor’s Journey
Q: Cliff, You’ve had an unusually successful life as a pioneer and inventor in the computer field. A CNN article describes one of your inventions as having “launched the text-messaging phenomenon.”
For years now, you’ve also had a strong spiritual life that includes the regular practice of meditation. Looking back on your life, can you identify any single event that helped shape who and what you’ve become today?
CK: I would say it all started with my exposure to the writings of John Lilly when I was a student at the University of Michigan in the 1970s. When I entered college I thought I’d become a mathematician. I had almost completed a math major when I read about the research Lilly was doing on dolphin communication. Reading about dolphins and listening to recordings of humpbacked whales, I became obsessed with whales and dolphins and switched my major from math to zoology.
John Lilly’s writings also introduced me to the idea that our beliefs can often shape the material reality we experience. Previously, I had become more or less a complete materialist – thinking that eventually physics would give us all of the answers. But Lilly’s concept was fascinating to me, and became a kind of springboard into exploring the spiritual realm.
Q: Given your interest in zoology and dolphin communication, why did you decide to become a computer scientist?
CK: A few years after college, I was about to head to California to offer my services to John Lilly, who was then building a computer interface for human/dolphin communication. But I realized I might need to have some computer skills, so I enrolled as a graduate student in the computer science department at Michigan State University. There I met Dr. John Eulenberg, a pioneer in the field of augmentative and alternative communication, and I became very interested in this new field.
Q: What is augmentative communication?
CK: Augmentative communication is the field of providing technology for people whose disabilities prevent them from using any of the more or less standard channels of communication. Some individuals are unable to speak intelligibly, type on a standard keyboard, or even use sign language – they have no way to express their thoughts or feelings. Augmentative communication provides such people with new technology that enables them to generate language, written and spoken.
Q: Was Michigan State a leader in this field?
CK: Yes, but at that time – the late 1970s – the field was very small. The computer industry was just blossoming and the idea that there were now tools that could really make a difference for people with disabilities was a very new concept. I just happened to be in the right place to be able to study under John Eulenberg, who was exploring and developing ways for people with disabilities to communicate by using the computer.
Q: What, if anything, did you do as a graduate student to develop new ways of helping people?
CK: At that time, all of the communications systems for people with disabilities required that they fit into an existing model. Essentially the message to them was, “OK here’s the system. Learn to live with it.”
I wanted to turn that around and say, “OK, what’s natural for you? What can you do with your body to generate a signal that the computer can respond to? Can you hit a switch? Can you control your head movements? For a person with cerebral palsy, for example, simply moving an arm to hit a specific key on a computer keyboard can be a major effort.
So I conceptualized a system in which we would attach electrodes to an individual that would detect the person’s nerve activity if a muscle moved – even if the person only barely started to try to move. The idea was to give people much greater control and to allow the individual to say, “OK, I want to use this particular movement as a signal.”
I was trying to put the focus back on the individual, and on what a person could actually do, instead of saying, “Here’s how you have to adapt to the system. Take it or leave it.”
Q: It sounds like you were trying to empower the individual. Is that true?
Q: Was Dr. Eulenberg supportive of what you were doing?
CK: Yes, he was.
Q: When did you create your first invention in this field, something that resulted in a patent?
CK: It was during my first job after I completed my doctoral studies. I took a job in Ohio where I designed and built what became known as the Liberator, a special purpose communication device for people who couldn’t speak. Pressing certain keys would activate either letters or “icons” that would bring up words and phrases, and the Liberator would speak the text that was generated.
While designing the Liberator, I came up with an invention that reduced the number of keys one had to push to activate the system, so that it would automatically switch between generating letters and icons. This made it a little easier for a disabled person to use the device.
Q: Was creating that invention an important moment in realizing what you could do with your life?
CK: Yes, and I became very excited about that aspect of it. Computer programming sounds like a very rote field but, in fact, it offers a lot of opportunities for creativity. When I’m writing a computer program, I may know from the start what I want it to do, and how people are going to use it, but then, as I go along, I find creative new ways to accomplish those goals. Sometimes I might end up creating a whole new way to interact with the technology — and that’s where the invention aspect comes into the picture.
Q: From what I’ve read about your inventions, all of them seem to have the common thread of making it significantly easier for people to communicate. Is that correct?
Q: Did the Liberator prove useful to people who couldn’t speak and needed augmentative communication?
CK: Yes. The Liberator did a lot of things that no other system had done before, and it was quite a step forward for non-speaking people at that time. But special purpose devices like the Liberator are very expensive to produce. By the time they’re produced, their technology is often out of date. I’ve always believed that a better approach for many people with disabilities would be to find ways to enable them to use mass market computer technology.
Q: Is it correct to say that you were able to put that belief into practice when you came up with the two inventions that have attracted major national attention: T9 and Swype?
CK: Yes – at least that was the original intent. Both inventions were initially conceived of as serving people with disabilities but in both instances, I could see that this new technology would also be useful to the cell phone mass market.
Q: Both T9 and Swype have had a very big impact on the cell phone industry and on cell phone use. Can you explain briefly what these two inventions accomplish?
CK: T9 was the technology that made it possible for cell phone users to type much more quickly and easily on a typical cell phone keypad with only 12 keys. T9 was not solely my invention. Two friends and I invented it during the mid-1990s.
The Swype technology came along in 2002. This was well before cell phones with touch-screens began to emerge. There were only a few specialized devices that had small touch-screens. But I believed that eventually people would want their phones to have touch-screens; when the iPhone finally came out, this vision came true. However, people found that it was much harder to tap out a message on a touch-screen keyboard than on a real keyboard. It took more time and typos were very common.
Swype solves these problems by allowing a person to spell words by gliding a finger across the keyboard — tracing a path through the letters of a word rather than tapping out each letter. The Swype software automatically inserts spaces between words and corrects spelling and other mistakes.
Overall, Swype is a much faster and more efficient way of inputting text on a cell phone. And most people just find it fun to use.
Q: I understand that in both cases you were able to sell these inventions – along with their patents – to large corporations?
CK: Yes. In 1999, AOL acquired Tegic Communications, the startup company we formed to market T9. Nuance Communications acquired Swype – the name we used for both the technology itself and the startup company that developed it – in 2011. Coincidentally, Nuance had previously acquired Tegic from AOL in 2007, so this was a kind of a homecoming for me.
Q: From what I’ve heard, the acquisition process involving large corporations can be very uncertain and stressful. Did being on the spiritual path when you were trying to find a corporate buyer for Swype make the process less stressful?
CK: The negotiations around the sale of a start up company can be very intense. Starting Tegic and developing T9 took us to the brink of bankruptcy quite early in the process, so yes, it was rather stressful.
But there was much more at stake with Swype than with T9. Eight years of effort had gone into developing Swype. We had succeeded in making a great product and had obtained valid patents. But because we were on a playing field with huge corporations looking for multi-million dollar profits, and with so many different factors at work, it was entirely possible that those eight years of work could have been pretty much for naught. Even in the final stages of the process, it was not at all clear what the outcome would be.
It was only because of the non-attachment and peace of mind that the spiritual path and practice of meditation have brought me, that I could look the possibility of failure squarely in the face and say, “OK, Master. If this is all a spiritual test and everything we’ve worked for goes nowhere, then that’s the lesson I need to learn.”
Really accepting that possibility in my heart took away so much stress. I’m not saying I was perfectly non-attached, but the process would have been much more stressful without the understanding and the faith to accept that possibility. And I was so grateful for that understanding and faith.
Q: After selling both inventions, were you able to fulfill your original intention of making the new technology available to people with disabilities?
CK: Yes. After the sale, I am once again working with Mark Illing, one of the software engineers who was involved in the development of T9 and who worked with me through many years to create Swype. We are now finally working together to try to create new applications of the T9 technology to assist people with disabilities.
Q: Will you be able to do the same with the Swype technology?
CK: Yes. In the acquisition of Swype, Nuance permanently retained close to half of the Swype staff, myself included. During the negotiations with Nuance, I was able to say, “I want to spend some time taking these technologies back to the disabilities realm. They’ve had a huge impact on the cell phone mass market but they were designed to help people with disabilities and I want to continue with that work as a significant part of what I do.”
Nuance graciously accepted those terms and, in terms of budget allocations, is supporting the disabilities work in a very significant way.
Q: Your commitment to helping people with disabilities suggests that dharma or “right action” is very important to you. In fact, your mentor, John Eulenberg, was quoted as saying that you “believe in doing good.” *
Do you see your commitment to helping people with disabilities as a form of dharma? Have you ever thought of it in these terms?
CK: I think a commitment to dharma is part of my nature, probably something I brought over from the past. There are times when in a specific situation it’s not always clear what is the dharmic thing to do, and trying to see the dharmic way has not always been easy.
Over time I have come to know that the best thing I can do is to try to become quiet, to meditate, and try to hear what Master is whispering. But yes, even if I didn’t think of my work as dharma at first (it just seemed like the best thing I could be doing with my time), I can now see it as a form of dharma.
Q: It seems that your commitment to dharma goes beyond helping people with disabilities and is a guiding principle in your life. One of your Swype associates describes you as having “incredible business ethics.”* Apparently he trusted you enough to agree to become CEO of Swype without a contract. Can you explain how that happened?
CK: As Swype grew, we needed someone to run the company and handle the marketing and sales to cell phone manufacturers, and it’s a tough job. I brought in Mike McSherry, who became Swype’s CEO. It was an intuitive decision. Though he’d never been a CEO, he felt like a kindred spirit and something told me, “Get this guy!” From the beginning there was such a level of trust between us that we simply shook hands and said, “OK, let’s do this together and we’ll be fair about how we handle this.”
There was no written contract and we didn’t even specify verbally, “OK, you’ll get this percentage of the company or such and such a salary.” Mike worked for several months before we could even understand the best way to define his relationship to the company.
Q: Everything seems to have worked out?
CK: Everything has worked out very well.
Q: You mentioned that your decision to hire Mike McSherry was “an intuitive decision.” What role has intuition played in your invention process?
CK: A big one! Whenever I experience problems or obstacles in my work, my practice is to focus very hard on resolving them. But it’s usually when I’m not focusing on the problem that, all of a sudden, boom! There’s the solution.
One example took place when we were first developing the Swype software. The devices we were using to run the software did not have much computational power – the software could have easily swamped them. I had to find a solution. For a few days I just didn’t know what to do. Then, while I was looking at a certain geometrical shape, the solution came in a flash.
Q: When you encountered these kinds of problems or obstacles, did you ever pray for solutions?
CK: Yes, I definitely appealed to God and Guru. Often when I’m stumped, I try to open up to the Divine to show me what I’m missing. Sometimes answers come in the middle of meditation.
Q: You mentioned that it took eight years to develop Swype. Was that because you encountered numerous problems or obstacles along the way?
CK: Yes, partly that and partly that I just had to keep refining and improving the system until it would “just work” for someone the first time he tried to use it. That’s not always easy to do.
One of Swype’s main features is that it’s “forgiving”— it corrects mistakes. But it took years to identify and correct potential mistakes. Often I’d be using the system and I’d try to enter a word and Swype wouldn’t get it right. I’d try to capture that case and use it to trace through thousands of lines of code to try to figure out where Swype went wrong and how to fix it. Delving into all of those individual cases took not only the most intense focus and concentration but also a lot of time.
Q: Looking ahead to the future. After you develop the disabilities version of Swype, what will you do next?
CK: Swype is currently 97% accurate in correcting mistakes. I’d like Swype to become 100% accurate. The technology that Nuance has available will help resolve the remaining ambiguities.
After that’s done, I’ll move on to something else.
Cliff Kushler is currently VP of Technology Innovation at Nuance Communications. In 1987, upon his return from earning a Ph.D. at the University of Tokyo, he joined Prentke Romich Co. to serve as Director of Research in the design and development of Augmentative and Alternative Communication (AAC) devices, which provide comprehensive communication support for individuals with motor impairments that prevent them from speaking intelligibly, using a standard keyboard effectively, or using sign language. This work led in 1995 to the co-invention, with his partners Martin King and Dale Grover, of the T9 text input technology in 1995 which was ultimately licensed on over 5 billion phones.
Following the acquisition of Tegic by America Online in 1999, Cliff left near the end of 2000 to pursue his own interests. In 2002, he partnered with Randy Marsden and, again building on work to facilitate computer access for individuals with severe motor impairments, began the development of the patented “Swype” continuous path-tracing text input technology now broadly licensed on Android and other smart-phone platforms. Cliff Kushler served as the CTO of Swype Inc. up until it was acquired by Nuance Communications in 2011. He has been awarded 33 US patents, as well as numerous corresponding internationally filed patents.
Cliff Kushler currently lives with his wife and son near Ananda Village and serves as a minister and light-bearer at Ananda Village.
* See: “The Man Who Invented the Keyboard Twice,” by Mark Milian, CNN Tech, January 5, 2011.