For some time, I have had an interest in the concept of tele-presence. During my mechanical engineering undergrad, many of the projects I undertook involving robotics and smart materials surrounded the idea of communicated the sense of touch over distance.
Early Mechanical Engineering projects of mine that intersected with HCI and telepresence
I was later inspired by numerous demonstrations of organic user interfaces I started to see on YouTube around the early 2010s. The included the G.H.O.S.T() interface labs and inFORM at the MIT media labs. The core interaction of transmitting shape as a means of a novel interaction was very appealing. Interaction with dynamic shape changing media was an HCI research question I wanted to explore during my time at the Georgia Tech HCI program.

Yeti and inForm, two previous projects of note that created and conduct user research with organic user interfaces
I also wanted to explore alternative media for organic user interfaces. Much of the prior art involved interfaces based on the tangible bits approach. This meant they were typically pixel-like sections that were actuated individually. However, because of the limitation of actuator size, this limited the complexity of geometries that could be created. 
I pondered about the possible off fluid based organic displayed controlled by pnuematics, hydraulics, or magnetism. Some of the prior art I reference can be found below.
The Living Liquid Display and SnOil, two previous displays that leverage a magnetic fluid as a means to communicate and facilitate interaction.
The biggest issue I kept running into with fluid based displays was they proved not effective for meaningful tangible interaction. They were appealing to observe and could form complex geometries, but medium like fero-fluid does not communicate solid structure through touch and is generally not a great user experience to touch.
I was stuck at an impasse trying to determine a material medium that would function like both a fluid and solid as needed.
This was the point at which I thought of using non-newtonian materials in the interface. Specifically something like oobleck that exhibited known shear-thickening properties. Oobleck is a mixture of water and cornstarch that exhibits properties of both a solid and liquid depending on whether of not force is applied to it at the time.
I did some research and found some limited resources concerning previous attempts to create oobleck interfaces. The prior art mainly employed the use of sound and vibration to activate the oobleck. The results were somewhat effective but really lacked any precise control over the oobleck.
.Fluid Display, a previous interface that employed oobleck and sound waves to create a novel interaction.
Introducing Feroobleck
I then opted to try magnetic control. By embedding magnetic particles in the oobleck, I believed I could have finer and more dynamic control over the oobleck. I called this combination Feroobleck. Over the next 6 months I worked to characterize it, improve it with additional chemical additives and pursue intellectual.
Demo-ing and Characterizing the new material
The next step was coming up with an user interface to test the oobleck medium. Ideally a magnetic matrix was desired. However, due to resource constraints on this independent project I opted to used a set of 4 solenoid valves with small neodyniums attached to them. Neodyniums were used instead of electro-magnets due to their extremely dense and strong magnetic fields.
The interface I came up with was designed to communicate fingertip touch between parties.

4 touch sensors were placed on a mouse shaped interface, when triggered, these would active relays that controlled solenoids with small neodyniums on there end. The solenoids would then deactivate, springing upwards to be in close proximity to a thin receptacle holding the oobleck. A user could then feel the shape of the oobleck change in specific locations to reflect the feeling of the person pressing there fingertips against the sensor area.
Currently, this interface is being used to test interaction with the material. I am attempting to see how well users can recognize activated areas of the material and identify which finger presses have been activated.
Results thus far have been been encouraging. Although it is still a somewhat messy medium users are able to identify activated lump areas of the feroobleck, an advantage of the material changing both shape and state when activated. There was some uncertainty expressed at points as oobleck will change state no matter which force is acting on it, so even if users pressed inactivated areas, they may experience the sensation of a solid forming under their fingertips.
Further testing will continue, but this was an encouraging start to pursuing effective mediums for organic shape changing displays.

Back to Top