Professor Seth Goldstein, a Carnegie Mellon computer science professor,
leads the project and was kind enough to answer some questions for G4tv.
G4tv.com: How did you and your researchers come up with the idea? Are there any solid points of inspiration is it a natural evolution from previous research?
Seth Goldstein: It was a little bit of both. Before working on Claytronics I was doing research in the area of molecular computing, which is how to build computers out of molecules. Essentially, in molecular electronics we are using the change in the shape of molecules to influence a computation. When the molecules take a different shape they have different electrical properties and you get a different circuit.
Seth (cont’d): One way we change the shape of the molecule is with electricity. So, I thought, what if instead of using the shape-change in a molecule to influence a computation, we do it the other way around? We do some computation and the result of that is to change the shape of the molecule.
That general idea becomes, can we develop a material and a programming methodology that allows us to change some physical characteristic of a material based on a computation. The general class of stuff that has that behavior is called programmable matter.
At about this time, about 4-5 years ago, there was a conference hosted by CRA (Computer Research Association) to look for a grand challenge in computer systems research, something like going to the moon that we could use to inspire new students and faculty in their research. Both Todd Mowry, also a professor at Carnegie Mellon, and I were at this conference and he was interested in improving communication. He thought that the programmable material idea would be perfect for a better way for people to communicate. We went to lunch and he proposed we start now instead of waiting for nanotechnology to catch up. And, then the project was born.
One of the goals of the project is work on a long range, out-there idea that would force us to re-think assumptions and come up with new designs and inspire people to have a common focus among their research.
G4tv.com: How many years away is what we saw in the concept video from the ETC?
Seth: I’m a firm believer that humans have a hard time, myself included, understanding exponentials. We’re on some exponential technology curve with Moore’s Law. Of course, there are some engineering challenges. Getting this all to work is going to be difficult, but the rate of progress is going to be very surprising.
There are a couple of things about that car video. Right now we’re not doing any work on how to produce color. If we can get stuff to move around and do the right thing then [color isn’t] hard. Also, in the video, the concept is that there is Computer Aided Design (CAD) program running on the catoms which has the model of the car. That’s what it used to reproduce the model of the car. And that program is running essentially on the Claytronics because when they’re pushing the window around it doesn’t just morph, it actually understands that it’s supposed to make the trunk smaller and the hood changes shape. So it’s running all that constraint solving.
There’s a difference between running a program that just gets it to form a shape and then running the CAD program on that. So, leaving aside the color and the CAD program running in a distributed fashion on the Claytronics which are hard problems in it of themselves, I think somewhere between 5 and 10 years. It could be sooner, it could be later, I’m not sure. In the meantime, we’re doing a lot of interesting research and I think we’ll have a lot of impact on the modular robotic and distributed programming communities.
My expectation is that sometime in the next couple of years we will demonstrate having 4 or 5 particles that can move from [flat on] a plane to a pyramid. That demonstration seems a long way from our video, but once that happens I think there will be a lot of researchers who work on this problem and then the exponential growth kicks in.
G4tv.com: Can you talk about the distributed computing aspect of the project?
Seth: I think [this is] the main research challenge. At first glance, it may appear that the hardware is the main challenge. And, while there are a certainly a lot of hard engineering problems and some interesting science that has to be solved, one can see point solutions to all the hardware problems: energy distribution, getting rid of the heat, how they stick together, how they move around. It’s really about integrating it and how they work together. But the distributed programming part, that’s a very, very serious challenge.
G4tv.com: So will the program be contained in another unit, like the table we saw in the video?
Seth: No, the program is in the units. That’s one of the key things about it. Each one of these units has a processor, some memory, and has the ability to store some energy. [They have] the ability to move around other units and stick to them.
G4tv.com: This makes us think of body cells. We have bone cells, muscle cells, etc. They are specialized for a specific purpose. Is that how these work?
Seth: Right now in the Claytronics project, we’re basically focusing on a system of homogeneous units. Of course, there’s a reason we have bones and muscles-that is a much better solution to the problem of dynamic 3D shapes. But, choosing what the right elements are at this point seems premature to me, so […] we’re simplifying the problem one way, intellectually, by saying all the units are the same.
G4tv.com: How did you and your researchers come up with the idea? Are there any solid points of inspiration is it a natural evolution from previous research?
Seth Goldstein: It was a little bit of both. Before working on Claytronics I was doing research in the area of molecular computing, which is how to build computers out of molecules. Essentially, in molecular electronics we are using the change in the shape of molecules to influence a computation. When the molecules take a different shape they have different electrical properties and you get a different circuit.
Seth (cont’d): One way we change the shape of the molecule is with electricity. So, I thought, what if instead of using the shape-change in a molecule to influence a computation, we do it the other way around? We do some computation and the result of that is to change the shape of the molecule.
That general idea becomes, can we develop a material and a programming methodology that allows us to change some physical characteristic of a material based on a computation. The general class of stuff that has that behavior is called programmable matter.
At about this time, about 4-5 years ago, there was a conference hosted by CRA (Computer Research Association) to look for a grand challenge in computer systems research, something like going to the moon that we could use to inspire new students and faculty in their research. Both Todd Mowry, also a professor at Carnegie Mellon, and I were at this conference and he was interested in improving communication. He thought that the programmable material idea would be perfect for a better way for people to communicate. We went to lunch and he proposed we start now instead of waiting for nanotechnology to catch up. And, then the project was born.
One of the goals of the project is work on a long range, out-there idea that would force us to re-think assumptions and come up with new designs and inspire people to have a common focus among their research.
G4tv.com: How many years away is what we saw in the concept video from the ETC?
Seth: I’m a firm believer that humans have a hard time, myself included, understanding exponentials. We’re on some exponential technology curve with Moore’s Law. Of course, there are some engineering challenges. Getting this all to work is going to be difficult, but the rate of progress is going to be very surprising.
There are a couple of things about that car video. Right now we’re not doing any work on how to produce color. If we can get stuff to move around and do the right thing then [color isn’t] hard. Also, in the video, the concept is that there is Computer Aided Design (CAD) program running on the catoms which has the model of the car. That’s what it used to reproduce the model of the car. And that program is running essentially on the Claytronics because when they’re pushing the window around it doesn’t just morph, it actually understands that it’s supposed to make the trunk smaller and the hood changes shape. So it’s running all that constraint solving.
There’s a difference between running a program that just gets it to form a shape and then running the CAD program on that. So, leaving aside the color and the CAD program running in a distributed fashion on the Claytronics which are hard problems in it of themselves, I think somewhere between 5 and 10 years. It could be sooner, it could be later, I’m not sure. In the meantime, we’re doing a lot of interesting research and I think we’ll have a lot of impact on the modular robotic and distributed programming communities.
My expectation is that sometime in the next couple of years we will demonstrate having 4 or 5 particles that can move from [flat on] a plane to a pyramid. That demonstration seems a long way from our video, but once that happens I think there will be a lot of researchers who work on this problem and then the exponential growth kicks in.
G4tv.com: Can you talk about the distributed computing aspect of the project?
Seth: I think [this is] the main research challenge. At first glance, it may appear that the hardware is the main challenge. And, while there are a certainly a lot of hard engineering problems and some interesting science that has to be solved, one can see point solutions to all the hardware problems: energy distribution, getting rid of the heat, how they stick together, how they move around. It’s really about integrating it and how they work together. But the distributed programming part, that’s a very, very serious challenge.
G4tv.com: So will the program be contained in another unit, like the table we saw in the video?
Seth: No, the program is in the units. That’s one of the key things about it. Each one of these units has a processor, some memory, and has the ability to store some energy. [They have] the ability to move around other units and stick to them.
G4tv.com: This makes us think of body cells. We have bone cells, muscle cells, etc. They are specialized for a specific purpose. Is that how these work?
Seth: Right now in the Claytronics project, we’re basically focusing on a system of homogeneous units. Of course, there’s a reason we have bones and muscles-that is a much better solution to the problem of dynamic 3D shapes. But, choosing what the right elements are at this point seems premature to me, so […] we’re simplifying the problem one way, intellectually, by saying all the units are the same.
Three magnetic-based 45mm diameter
planar catoms. Each catom has 24 magnets which can be energized under
the control of the on-board processor. Catoms move by coordinating with
their neighbor and energizing the appropriate magnets to pull each
other together.
G4tv.com: So they’re all homogeneous in physical makeup, but do they all have the same program?
Seth: Philosophically, if you want to think about the cell analogy, they differentiate based on the program they are running. They probably will all get sent the same program because it’s just easier and then each one will run particular pieces of it. One analogy there is that every cell has the same DNA, but in some cells some of [the DNA] is activated and some isn’t.
[Each one] has the same program, but then based on where they’re located and what function they need to perform they’re executing a different piece of it.
G4tv.com: Like stem cells?
Seth: Yes, except that stem cells do physically differentiate and then become a nerve cell, for example. But, each Claytronics unit is hardware and can't change that way. Instead, the software changes. It’s more malleable. The biological analogy is a little worrisome, because it’s a little inexact and maybe a little scary. But, Claytronics don’t reproduce. They don’t change their physical shape. They have much less utility than a cell does.
G4tv.com: So there’s no worry of robots eventually taking us over based on this project?
Seth: This project won’t be involved, but that is certainly an open question. <laughs>
G4tv.com: The video mentions the interlocking version not requiring any power to maintain connections once they are made. How could this property be utilized?
Seth: Let’s say, it's a long way in the future and you’re living in a small apartment in New York. Instead of having a chair, and a table, and a couch and a bed cramming up your apartment, you just have Claytronics.
When you want to go to sleep, it’s a bed. When you have guests over, it’s a big table. When you’re eating on your own, it’s like a TV stand. Once it’s configured, you don’t want to have it be using energy to stay in that shape.
G4tv.com: So theoretically, would Claytronics be able to emulate different materials? Could you emulate wood and then a mattress with the same system?
Seth: General-purpose programmable matter might be able to do this, but there’s a reason why jet engines are made out of a particular kind of [metal] because they’ve got to be incredibly heat resistant and they take huge stress. We’re going to be able to make something that looks like a jet engine, but it’s not going to be a jet engine.
It might, though, be able to look like it and feel like it. So, if the particles are small enough, you can simulate texture. The way you know what something feels like is not by just putting your finger on it, but it’s actually dragging your finger over it. So, you’d have a hard time telling the difference between a tile and some silk if they were at the same temperature and you have just put your finger on it, but as soon as you move it you could feel [the difference].
I believe that if we can’t emulate the feel of the material like skin versus a pair of jeans, we will have failed in the long run. I’m talking about the really long run, not the first 10 years. So, my expectation is that we will be able to do sound, vision, and touch. Taste, I don’t believe we’ll do and smell seems a little far off, but 3 out of the 5 senses.
G4tv.com: But it could be useful in teaching about jet engines?
Seth: Or designing them. Maybe it won’t be a jet engine, but it still has moving parts and you can see how it spins and you could put it in a wind tunnel. You’d have to adjust your expectations, but as a revolution in the design industry, it will be amazing, because you’ll have this, essentially, 3D CAD-enabled clay.
Instead of having to draw it in 2D and look at it in perspective and watch a simulation of how it moves in your CAD tool, you’ll actually have it in front of you. That’s what that car video was trying to get across.
G4tv.com: Will Claytronics offer the same resistance as a solid object or will it be easy to push the units out of place?
Seth: So, there are two things that come into play here. There are the strength and material properties of the individual units and then there are the kinds of adhesion forces that they can generate.
The way we’re planning on making them now, things would be pretty fragile. You wouldn’t want to use it as your car door because if someone punches it their fist would go right through it.
On the other hand, these things are running a program all the time and so you can imagine that they can take active measures to retain their cohesion. And I think that there is some very interesting research there.
Seth: Philosophically, if you want to think about the cell analogy, they differentiate based on the program they are running. They probably will all get sent the same program because it’s just easier and then each one will run particular pieces of it. One analogy there is that every cell has the same DNA, but in some cells some of [the DNA] is activated and some isn’t.
[Each one] has the same program, but then based on where they’re located and what function they need to perform they’re executing a different piece of it.
G4tv.com: Like stem cells?
Seth: Yes, except that stem cells do physically differentiate and then become a nerve cell, for example. But, each Claytronics unit is hardware and can't change that way. Instead, the software changes. It’s more malleable. The biological analogy is a little worrisome, because it’s a little inexact and maybe a little scary. But, Claytronics don’t reproduce. They don’t change their physical shape. They have much less utility than a cell does.
G4tv.com: So there’s no worry of robots eventually taking us over based on this project?
Seth: This project won’t be involved, but that is certainly an open question. <laughs>
G4tv.com: The video mentions the interlocking version not requiring any power to maintain connections once they are made. How could this property be utilized?
Seth: Let’s say, it's a long way in the future and you’re living in a small apartment in New York. Instead of having a chair, and a table, and a couch and a bed cramming up your apartment, you just have Claytronics.
When you want to go to sleep, it’s a bed. When you have guests over, it’s a big table. When you’re eating on your own, it’s like a TV stand. Once it’s configured, you don’t want to have it be using energy to stay in that shape.
G4tv.com: So theoretically, would Claytronics be able to emulate different materials? Could you emulate wood and then a mattress with the same system?
Seth: General-purpose programmable matter might be able to do this, but there’s a reason why jet engines are made out of a particular kind of [metal] because they’ve got to be incredibly heat resistant and they take huge stress. We’re going to be able to make something that looks like a jet engine, but it’s not going to be a jet engine.
It might, though, be able to look like it and feel like it. So, if the particles are small enough, you can simulate texture. The way you know what something feels like is not by just putting your finger on it, but it’s actually dragging your finger over it. So, you’d have a hard time telling the difference between a tile and some silk if they were at the same temperature and you have just put your finger on it, but as soon as you move it you could feel [the difference].
I believe that if we can’t emulate the feel of the material like skin versus a pair of jeans, we will have failed in the long run. I’m talking about the really long run, not the first 10 years. So, my expectation is that we will be able to do sound, vision, and touch. Taste, I don’t believe we’ll do and smell seems a little far off, but 3 out of the 5 senses.
G4tv.com: But it could be useful in teaching about jet engines?
Seth: Or designing them. Maybe it won’t be a jet engine, but it still has moving parts and you can see how it spins and you could put it in a wind tunnel. You’d have to adjust your expectations, but as a revolution in the design industry, it will be amazing, because you’ll have this, essentially, 3D CAD-enabled clay.
Instead of having to draw it in 2D and look at it in perspective and watch a simulation of how it moves in your CAD tool, you’ll actually have it in front of you. That’s what that car video was trying to get across.
G4tv.com: Will Claytronics offer the same resistance as a solid object or will it be easy to push the units out of place?
Seth: So, there are two things that come into play here. There are the strength and material properties of the individual units and then there are the kinds of adhesion forces that they can generate.
The way we’re planning on making them now, things would be pretty fragile. You wouldn’t want to use it as your car door because if someone punches it their fist would go right through it.
On the other hand, these things are running a program all the time and so you can imagine that they can take active measures to retain their cohesion. And I think that there is some very interesting research there.
