An Interview with Professor Seth Glodstein

Professor Seth Goldstein, a Carnegie Mellon computer science professor, leads the project and was kind enough to answer some questions for G4tv.

G4tv.com:  How did you and your researchers come up with the idea? Are there any solid points of inspiration is it a natural evolution from previous research?

Seth Goldstein: It was a little bit of both. Before working on Claytronics I was doing research in the area of molecular computing, which is how to build computers out of molecules. Essentially, in molecular electronics we are using the change in the shape of molecules to influence a computation. When the molecules take a different shape they have different electrical properties and you get a different circuit.

Seth (cont’d): One way we change the shape of the molecule is with electricity.  So, I thought, what if instead of using the shape-change in a molecule to influence a computation, we do it the other way around? We do some computation and the result of that is to change the shape of the molecule.

That general idea becomes, can we develop a material and a programming methodology that allows us to change some physical characteristic of a material based on a computation.  The general class of stuff that has that behavior is called programmable matter.

At about this time, about 4-5 years ago, there was a conference hosted by CRA (Computer Research Association) to look for a grand challenge in computer systems research, something like going to the moon that we could use to inspire new students and faculty in their research.  Both Todd Mowry, also a professor at Carnegie Mellon, and I were at this conference and he was interested in improving communication.  He thought that the programmable material idea would be perfect for a better way for people to communicate.   We went to lunch and he proposed we start now instead of waiting for nanotechnology to catch up.  And, then the project was born.

One of the goals of the project is work on a long range, out-there idea that would force us to re-think assumptions and come up with new designs and inspire people to have a common focus among their research.
G4tv.com:  How many years away is what we saw in the concept video from the ETC?

Seth: I’m a firm believer that humans have a hard time, myself included, understanding exponentials. We’re on some exponential technology curve with Moore’s Law. Of course, there are some engineering challenges. Getting this all to work is going to be difficult, but the rate of progress is going to be very surprising.

There are a couple of things about that car video. Right now we’re not doing any work on how to produce color. If we can get stuff to move around and do the right thing then [color isn’t] hard.  Also, in the video, the concept is that there is Computer Aided Design (CAD) program running on the catoms which has the model of the car. That’s what it used to reproduce the model of the car. And that program is running essentially on the Claytronics because when they’re pushing the window around it doesn’t just morph, it actually understands that it’s supposed to make the trunk smaller and the hood changes shape. So it’s running all that constraint solving.

There’s a difference between running a program that just gets it to form a shape and then running the CAD program on that.  So, leaving aside the color and the CAD program running in a distributed fashion on the Claytronics which are hard problems in it of themselves, I think somewhere between 5 and 10 years. It could be sooner, it could be later, I’m not sure. In the meantime, we’re doing a lot of interesting research and I think we’ll have a lot of impact on the modular robotic and distributed programming communities.

My expectation is that sometime in the next couple of years we will demonstrate having 4 or 5 particles that can move from [flat on] a plane to a pyramid. That demonstration seems a long way from our video, but once that happens I think there will be a lot of researchers who work on this problem and then the exponential growth kicks in.

G4tv.com:  Can you talk about the distributed computing aspect of the project?

Seth: I think [this is] the main research challenge. At first glance, it may appear that the hardware is the main challenge.  And, while there are a certainly a lot of hard engineering problems and some interesting science that has to be solved, one can see point solutions to all the hardware problems: energy distribution, getting rid of the heat, how they stick together, how they move around. It’s really about integrating it and how they work together. But the distributed programming part, that’s a very, very serious challenge.

G4tv.com:  So will the program be contained in another unit, like the table we saw in the video?

Seth: No, the program is in the units. That’s one of the key things about it. Each one of these units has a processor, some memory, and has the ability to store some energy. [They have] the ability to move around other units and stick to them.

G4tv.com:  This makes us think of body cells. We have bone cells, muscle cells, etc. They are specialized for a specific purpose. Is that how these work?

Seth: Right now in the Claytronics project, we’re basically focusing on a system of homogeneous units. Of course, there’s a reason we have bones and muscles-that is a much better solution to the problem of dynamic 3D shapes. But, choosing what the right elements are at this point seems premature to me, so […] we’re simplifying the problem one way, intellectually, by saying all the units are the same.

Three magnetic-based 45mm diameter planar catoms.  Each catom has 24 magnets which can be energized under the control of the on-board processor.  Catoms move by coordinating with their neighbor and energizing the appropriate magnets to pull each other together.

G4tv.com:  So they’re all homogeneous in physical makeup, but do they all have the same program?

Seth: Philosophically, if you want to think about the cell analogy, they differentiate based on the program they are running. They probably will all get sent the same program because it’s just easier and then each one will run particular pieces of it. One analogy there is that every cell has the same DNA, but in some cells some of [the DNA] is activated and some isn’t.

[Each one] has the same program, but then based on where they’re located and what function they need to perform they’re executing a different piece of it.

G4tv.com:  Like stem cells?

Seth: Yes, except that stem cells do physically differentiate and then become a nerve cell, for example. But, each Claytronics unit is hardware and can't change that way. Instead, the software changes. It’s more malleable. The biological analogy is a little worrisome, because it’s a little inexact and maybe a little scary. But, Claytronics don’t reproduce. They don’t change their physical shape. They have much less utility than a cell does.

G4tv.com:  So there’s no worry of robots eventually taking us over based on this project?

Seth: This project won’t be involved, but that is certainly an open question. <laughs>

G4tv.com:  The video mentions the interlocking version not requiring any power to maintain connections once they are made. How could this property be utilized?

Seth: Let’s say, it's a long way in the future and you’re living in a small apartment in New York. Instead of having a chair, and a table, and a couch and a bed cramming up your apartment, you just have Claytronics.

When you want to go to sleep, it’s a bed. When you have guests over, it’s a big table. When you’re eating on your own, it’s like a TV stand. Once it’s configured, you don’t want to have it be using energy to stay in that shape.

G4tv.com:  So theoretically, would Claytronics be able to emulate different materials? Could you emulate wood and then a mattress with the same system?

Seth: General-purpose programmable matter might be able to do this, but there’s a reason why jet engines are made out of a particular kind of [metal] because they’ve got to be incredibly heat resistant and they take huge stress. We’re going to be able to make something that looks like a jet engine, but it’s not going to be a jet engine.

It might, though, be able to look like it and feel like it. So, if the particles are small enough, you can simulate texture. The way you know what something feels like is not by just putting your finger on it, but it’s actually dragging your finger over it. So, you’d have a hard time telling the difference between a tile and some silk if they were at the same temperature and you have just put your finger on it, but as soon as you move it you could feel [the difference].

I believe that if we can’t emulate the feel of the material like skin versus a pair of jeans, we will have failed in the long run. I’m talking about the really long run, not the first 10 years. So, my expectation is that we will be able to do sound, vision, and touch. Taste, I don’t believe we’ll do and smell seems a little far off, but 3 out of the 5 senses.

G4tv.com:  But it could be useful in teaching about jet engines?

Seth: Or designing them. Maybe it won’t be a jet engine, but it still has moving parts and you can see how it spins and you could put it in a wind tunnel. You’d have to adjust your expectations, but as a revolution in the design industry, it will be amazing, because you’ll have this, essentially, 3D CAD-enabled clay.

Instead of having to draw it in 2D and look at it in perspective and watch a simulation of how it moves in your CAD tool, you’ll actually have it in front of you. That’s what that car video was trying to get across.

G4tv.com:  Will Claytronics offer the same resistance as a solid object or will it be easy to push the units out of place?

Seth: So, there are two things that come into play here. There are the strength and material properties of the individual units and then there are the kinds of adhesion forces that they can generate.

The way we’re planning on making them now, things would be pretty fragile. You wouldn’t want to use it as your car door because if someone punches it their fist would go right through it.

On the other hand, these things are running a program all the time and so you can imagine that they can take active measures to retain their cohesion. And I think that there is some very interesting research there.

Research Paper

Check out the latest research paper on Claytronics-Modular robotics to its new Extreme. Totally based on IEEE pattern..

http://www.mediafire.com/file/61qad91bh9ceb1l/Claytronics.pdf

CLAYTRONICS ppt.

Download

Download the complete document here:

http://www.mediafire.com/file/142koa88aq4awqu/CLAYTRONICS.pdf

Envisioning the Future

          Backed by the microchip manufacturer Intel, first generation catoms, measuring 4.4 centimetres in diameter and 3.6 centimetres in height have already been created. The goal is to eventually produce catoms that are one or two millimetres in diameter-small enough to produce convincing replicas. It's not just a problem of building tiny robots, but figuring out how to power them, to get them to stick together and to coordinate and control millions or billions of them. These catoms, which are ringed by several electromagnets, are able to move around each other to form a variety of shapes containing rudimentary processors and drawing electricity from a board that they rest upon. So far only four catoms have been operated together. The plan though is to have thousands of them moving around each other to form whatever shape is desired and to change colour, also as required. 

Five years from now, the DPR researchers expect to have working ensembles of catoms that are close to spherical in shape. These catoms still will be large enough that no one will confuse a replica with the real thing (for that, catoms will probably have to shrink to less than a millimetre in diameter). But the catoms will be sufficiently robust that researchers can experiment with a variety of shapes, test hypotheses about ensemble behaviour, and begin to envision where the technology might lead within a decade or two.

While the potential applications of dynamic physical rendering are exciting, the work being done at Intel Research Pittsburgh and Carnegie Mellon University has broader implications. At its core, the research involves learning to design, power, program and control a densely packed set of microprocessors. These are similar to the key challenges facing the computer industry today. As a result, the DPR research is likely to produce new insights and technologies that could influence the future of computing and communications.

If, in 1960, someone had suggested that one day you could buy a million transistors for a penny, the prediction would have seemed outlandish. But today Intel sells transistors for less than a micro cent, thanks to the continuing technology advances predicted by Moore's Law. It's not unreasonable to predict that one day far in the future; it may be possible to buy a million catoms for a penny.

But dynamic physical rendering could become viable long before Moore's Law drives down the cost of a catom to a micro cent. Even if catoms could be produced for a dollar each, some visualization applications might be economically viable. Certain other applications, such as programmable antennas, could be attractive even if a catom sold for tens or hundreds of dollars.

Whatever the cost, building catoms that are one millimetre in diameter-small enough to create convincing replicas-will be a difficult engineering challenge. But given current industry knowledge and the state of the art of silicon technology, it is not outside the realm of possibility. The challenge lies less in developing new technology than in bringing together a number of research areas in which the industry has made tremendous technical progress in the last decade.

Application of Claytronics/DPR

          The potential applications of dynamic physical rendering are limited only by the imagination. Following are a few of the possibilities:

> Medicine: A replica of your physician could appear in your living room and perform an exam. The virtual doctor would precisely mimic the shape, appearance and movements of your "real" doctor, who is performing the actual work from a remote office.

> Disaster relief: Human replicas could serve as stand-ins for medical personnel, firefighters, or disaster relief workers. Objects made of programmable matter could be used to perform hazardous work and could morph into different shapes to serve multiple purposes. A fire hose could become a shovel, a ladder could be transformed into a stretcher.

 

>  Entertainment: A football game, ice skating competition or other sporting event could be replicated in miniature on your coffee table. A movie could be recreated in your living room, and you could insert yourself into the role of one of the actors.

> 3D physical modeling: Physical replicas could replace 3D computer models, which can only be viewed in two dimensions and must be accessed through a keyboard and mouse. Using claytronics, you could reshape or resize a model car or home with your hands, as if you were working with modeling clay. As you manipulated the model directly, aided by embedded software that's similar to the drawing tools found in office software programs, the appropriate computations would be carried out automatically. You would not have to work at a computer at all; you would simply work with the model. Using claytronics, multiple people at different locations could work on the same model. As a person at one location manipulated the model, it would be modified at every location. 

Capabilities of Catoms

While catoms will be simple in design, each will have four capabilities:

Ø  Computation: Researchers believe that catoms could take advantage of existing microprocessor technology. Given that some modern microprocessor cores are now under a square millimetre, they believe that a reasonable amount of computational capacity should fit on the several square millimetres of surface area potentially available in a 2mm-diameter catom.

Ø  Motion: Although they will move, catoms will have no moving parts. This will enable them to form connections much more rapidly than traditional microrobots, and it will make them easier to manufacture in high volume. Catoms will bind to one another and move via electromagnetic or electrostatic forces, depending on the catom size.
              Imagine a catom that is close to spherical in shape, and whose perimeter is covered by small electromagnets. A catom will move itself around by energizing a particular magnet and cooperating with a neighbouring catom to do the same, drawing the pair together. If both catoms are free, they will spin equally about their axes, but if one catom is held rigid by links to its neighbours, the other will swing around the first, rolling across the fixed catom's surface and into a new position. Electrostatic actuation will be required once catom sizes shrink to less than a millimetre or two. The process will be essentially the same, but rather than electromagnets, the perimeter of the catom will be covered with conductive plates. By selectively applying electric charges to the plates, each catom will be able to move relative to its neighbours.

Ø  Power: Catoms must be able to draw power without having to rely on a bulky battery or a wired connection. Under a novel resistor-network design the researchers have developed, only a few catoms must be connected in order for the entire ensemble to draw power. When connected catoms are energized, this triggers active routing algorithms which distribute power throughout the ensemble.

Ø  Communications: Communications is perhaps the biggest challenge that researchers face in designing catoms. An ensemble could contain millions or billions of catoms, and because of the way in which they pack, there could be as many as six axes of interconnection.
            Another unique feature of catom networks is that catoms are homogeneous. Thus, unlike cell phones or other communications devices, the identity of an individual catom is sometimes (but not always) unimportant. An application is more likely to care about routing a message to the catoms comprising a specific physical part of an ensemble (for instance, the catoms comprising a "hand") rather than sending the same message to specific catoms based on their serial numbers. Furthermore, catoms may be in motion periodically, as the shape of the ensemble changes.

Ø  Creating the replica:   At a high level, there are two steps :

·         Capturing a moving, three-dimensional image and

·         rendering it as a physical object.

Researchers at Carnegie Mellon University also are exploring 3D image capture, in the Virtualized Reality project. They have developed technology that points a set of cameras at an event and enables the viewer to virtually fly around and watch the event from a variety of positions. The DPR researchers believe a similar approach could be used to capture 3D scenes for use in creating physical, moving 3D replicas.

Replicas will be created from Catoms. Catoms can be formed into different shapes, and it can change color, through light-emitting diodes on its surface. Embedded photo cells will enable it to sense light, so that a human replica can "see." Catoms might even simulate the texture of the person or object being replicated. A replica will have computing capabilities, but these will be accessed through touch, voice, or another natural interface rather than a keyboard or mouse. Catoms will be as close to spherical as possible to support multiple packing densities.