Haptic Feedback

From CS294-84 Spring 2013
Jump to: navigation, search

Valkyrie Arline Savage, PhD - 3/9/2013 23:18:15

The haptic output devices in this week’s papers are intriguing. I have to start with the weird one.

The rubbing and tapping paper. I just don’t even know. I see the benefit of having a “human-like” notifications interface for machines, but this paper reads as though the authors were so deep in their research that they didn’t take time to think about its applicability. At all. They spent a long time suggesting that this device could be used for mobile interaction, but then they built prototypes that were >1 foot long. They even stated that this was a pretty inherent limitation of the actuation needed: participants couldn’t distinguish things when they were moved over too small an amplitude. Then there is the wonderful sentence about “personal tactile ringtones”. That just sounds... well, lewd. Anyway, at least they did a good job on their evaluation of their preposterous interface. I must admit that I laughed aloud at several points during reading this paper...

The TeslaTouch paper and the MudPad paper were both kind of awesome, and I feel like they go together as a unit, but I guess I will treat them separately for the sake of the presenter. The TeslaTouch paper creates a dynamic electrostatic coupling between a user’s moving finger and the touch surface: this leads to haptic feedback which users described as between friction and vibration. The main limitation of the approach is that it gives a uniform texture over the whole surface of the table, which has interesting consequences for e.g. multiuser scenarios. The authors suggest that “Although TeslaTouch can only provide one tactile signal to the entire surface, only moving fingers feel the tactile feedback. By carefully designing interactive sequences so that out of all fingers touching the surface only one moves at a time, we can create an illusion of localized tactile feedback,” which personally I think is total horse-poop. I do think that electrovibration is pretty awesome, and that they did a good job of characterizing what could be done with such an interface (and how it would be perceived by users).

The MudPad is built out of crazy magnetorheological fluid controlled by electromagnets, and unlike the TeslaTouch it allows for localized haptic feedback rather than feedback over the whole surface. They also in their prototype consider the difference that this makes: they allow a user to explore the interface’s haptics without interacting with it, which is smart! It’s sad that the current implementation is limited to top-projected graphics, but presumably there will someday be transparent ferrofluids. One thing that I thought was particularly interesting, though not a contribution of this paper, was the secure touchpad. I want to try this vs. the TeslaTouch and see which one feels more like touching a real object.. and which one actually leads to eyes-free interactability. I’m not certain that either of them would...


Ben Zhang, PhD - 3/10/2013 14:14:58

  • TeslaTouch (Bau)

This work explores the possibility of using electrovibration as haptic feedback for user interaction. Though electrovibration is not new, there seems to be few formal user studies on their application of haptic feedback (mostly because they were using dense arrays of metal pins which makes it hard to be integrated to any tracking/display system.

This paper starts with a natural integrated version - exciting the electrode layer of a commodity touch panel. After briefly discussing their apparatus, grounding strategy and safety issue, they state their system implementation details. One major problem with these novel haptic feedback is that some feelings are too subjective to be justified. They conduct extensive experiments to explore the design space - voltage, frequency, detection threshold, etc. The overall rating seems promising, and as authors have pointed out, electrovibration offer some unique chances over mechanical simulation, such as spatial uniformity, frequency-independent damping, reliability, etc. Several example applications are then discussed which demonstrate the TeslaTouch's capability.

Generally I like this approach of augmenting interaction with haptic feedback, which is even able to simulate different texture feelings. With nice integration of tracking/display, the device just looks fantastic. One issue regarding the paper, if I am understanding correctly, is that this feeling will only exist when relative position change occurs, more like "friction". Then would there be feedback when tapping? As their paper indicates "only digits in motion perceive this effect [vibration or friction]". But should clicking (from hover to touch) being motion too?

Though they have a lot of evaluating human's response to different parameter settings, a larger scale study might be helpful for boosting the adoption. Besides, this paper mentions that an amplitude of 8 Vpp would also work, however, their prototype and evaluations are based on ~100 Vpp level. This isn't a major issue when you can plug most of your screen device into wall-plugs; it does impede the possibility of mobility. So naturally I am interested about the performance in those scenarios.


  • SoundTouch (Li)

SoundTouch is a system that utilize the voice coil motor to implement tapping and rubbing tactile feedback. From the paper's content, they discusses their adoption of coil motor from cheap disk drive, and how they choose the right material for tapping/rubbing interface. As the same with TeslaTouch paper, extensive work of evaluation is necessary to define human's perception of the haptic feeling. The ability of differentiating and identifying is crucial especially when there is no visual cues available. The general feedback from participants is that such tactile feedback is more appropriate for silent notification (in comparison with vibration), and they sometimes feel like being interacting with other people.

Admittedly, the tapping and rubbing feedback is different from most approach of providing merely vibration. However, as for the soundTouch system, several concerns are unavoidable. First, their prototype largely limited the mobility/portability. Who would carry such a large device? Though mobile phones do have problem of vibration too aggressively, I believe their intensity can be adjusted (just not sure why this isn't the case for current phone implementation). More inherently, the tapping and rubbing do require movement of large objects, which will definitely be the limitation; on the other hand, the phone's vibration may have less freedom, but they exhibit enough bandwidth for notification. I am not criticizing the overall tapping/rubbing haptic, and potentially someday, either simulated or well-engineered system can be made also portable enough to support it. But it just didn't seem right with their approach. Second, I didn't see much discussion of mapping these actions into appropriate interactions. Though equipped with large freedom, it isn't clear to me how different tapping and rubbing can be directly used for users. Customized message can be conveyed through, though, but not natural to me, at least.

  • MudPad (Jansen)

MudPad uses magnetorheological fluid whose viscosity can be controlled by the magnetic field. The response time is fast enough (typical 2ms); and the bottleneck of resolution is the magnets (which needs to be strong enough, thus large). Use cases like music controller, virtual keyboard can be then augmented with tactile feedback.

There are numerous papers related to this project; I haven't found any of them did a user study about people's reception of this technology. While the use of magnetorheological fluid is interesting, little discussions are drawn upon the integration with other display/tracking method. Their prototype uses top projector due to the magnets underneath. Also the problem with scalability (large screen, more magnets) and low resolution are nontrivial for real use.

Overall, I feel that they don't have a complete discussion in any of their papers which articulate the issues related with interaction.


elliot nahman - 3/11/2013 20:52:17

TeslaTouch

TeslaTouch is a novel was of providing tactile feedback on surfaces. Unlike conventional haptic feedback, which utilize motors to cause physical vibrations, it employs electrovibrations. Electrovibrations are caused by an oscillating electric field in the surface which a user can sense while dragging a finger over the display. In this paper, the authors categorized the different sensations of touch users felt at different voltages/frequency combinations. They also added TeslaTouch to a multitouch display and compared performance against a more tradition, motor actuated, haptic feedback system.

I find the idea of TeslaTouch rather intriguing, that you could add different touch sensations to an otherwise static display. However, the limitation that a finger must be in motion seems a rather large limitation. Thinking about apps I use on my phone, the dominant haptic feedback experience is for single touches such as clicking. I cannot think of instances where I feel haptic feedback for dragging. Although there may be good instances when adding tactile sensation to dragging is desirable, it seems like the dominant interaction is not to. Currently, with motor actuated haptic feedback, there is no reason one cannot add it to dragging, but it is not commonly done. Swype for instance does not provide haptic feedback as you slide over keys, even though it would if you just clicked on one. I am curious as to why this is; is it just because of the one dimensionality of motor actuated haptic feedback? Or it is just not generally seen as a necessary or useful form of feedback? Basically, although neat, I question how useful it really is.

It seems that the real problem is touch screens still basically offer one dimensionality of touch as compared with physical interfaces. With a keyboard, you can run/rest your fingers on it providing one sense of tactile feedback. Pressing down offers a second and constitutes a click. Adding any haptic feedback to a touchscreen gives some indication of where you are touching/clicking, but not the exploratory first-touch you get with a physical device. If resting and having sensation were an opportunity with TeslaTouch, it seems like one could almost mimic this experience which would be really awesome.


SoundTouch SoundTouch is an alternate touch feedback mechanism making it possible to perceive taps or rubs. The authors built an actuator device out of a hard drive offset motor and build a series of hammer like devices to either strike the user or rub the user. In this way, they can provide feedback signals to the user. They then conducted user studies to understand if users could identify different frequencies of taps and rubs.

I find it strange that the authors seem to mention alerts and the potential to use this device as a way to communicate short messages. It seems like this is much more useful as a haptic feedback strategy for the user to experience along with other feedback on their device. Not a standalone system which would require constant physical contact with the device. Implementing this as part of a phone or game controller for when the user is holding the device, makes sense. Some standalone device that the user would have to wear or otherwise be in contact with seems too specialized for such a low bandwidth form of communication. The steering wheel example seems strange since people move their hands about the steering wheel and vibrations from that device generally already give feedback about the road; adding more tactile sensations to it seems out of place. The chair also is bizarre, especially since the chair would come into contact with different parts of the body than they tested, parts of the body that would probably be less responsive to subtle touches and therefor have even lower bandwidth for communication.


MudPad More similar to TeslaTouch, this is a tactile feedback mechanism intended to sit over a multitouch display and provide haptic feedback about certain areas of the screen. It employs a magnetorheologic fluid which can align itself under the influence of a magnetic field. By doing so, a surface with texture can be created. Unfortunately, it is an opaque substance and therefor can only be projected on rather than sit on top of a display.

This system addresses the issues I raised in TeslaTouch, allowing one to explore a surface with a “hover state”. Unfortunately, the fact that it requires a projection from above severely limits its applications. I also wish there were some discussion of just how many degrees of surface differentiation they could create. It sounds like using PWM, there is some potential for a variety, but it is not really discussed just how many levels are perceivable to a user.


arie meir - 3/11/2013 22:57:59

Teslatouch presents an adaptation of existing electrovibration technology as an enabling technology for user-interface applications. Compared to mechanical vibrations, it has clear advantages such as the lack of moving parts, silent operation and flexibiilty in applying to various surfaces. The disadvantage of electroverbration seems to be in its ability to provide feedback only during motion, i.e. it is not suitable for a typical click-like interactions with touch-surfaces. The accumulation of charge on the top of the surface might produce minor sparks (static electricity) and this could be a potential problem as it means that the user would experience clicks when the charge runs to ground through a grounding strap, and also the strap itself poses a limitation for a casual use. The study is well motivated and the user experience data seems to be thorough, although I would be curious to understand the behavior of the system without the grounding wrist strap over time as charge accumulates. Perhaps a sensing mechanism could remove the charge from the touch surface to account for static accumulation.

SoundTouch presents an interesting haptic feedback device which can translates an arbitrary sound signal into mechanical vibration by using a hard-drive actuator. Two types of of feedback are implemented : rubbing and tapping. The user experience study seems systematic in checking how are the different modalities percieved. Would be interesting to see if higher bandwidth signals could be communicated - after all - the system can transduce audio signals. In the technical aspect, i would be curious how small could this system be made, after all, the typical haptic actuators are as small as 3-5mm, it would be hard to compete with this., but the system could definitely be valuable in non ultra-mobile modalities.


The magnetic fluid based interface is interesting due to the underlying physical principles. By controlling the local magnetic field, the physical properties of the smart MR fluid are modulated and the liquid's viscosity can be altered. The "analog" nature of the device allows one to develop extremely rich interface primitives such as a simulated button. On the limitation side, the necessity to have a strong magnet could be a limiting factor in terms of size. Also, I wasn't sure how exactly the projector gets the image on the touch screen from behind a layer of electromagnets.


Overall, it seems that the community understands that the proliferation of touch screens which are typically a single-direction flow channel (input) calls for a variety of possible feedback mechanisms which would make the interfaction experience more dynamic, realistic and controlled. While a single mechanism cannot fit all applications, a philosophy of "to each his own" should provide enough basic elements to address the different needs various applications might have


Joey Greenspun - 3/11/2013 23:19:56

Tapping and Rubbing: This paper proposes a device called soundTouch that can apply novel tactile input to a user via either tapping or rubbing. A main goal of theirs was to apply vibrational (tap) inputs with frequencies of less than 20 hz which cannot be done using current hardware. One thing this study does a great job with is identifying if users can actually use this device effectively. They performed studies to ensure that users could distinguish the difference between various intensities and frequencies of tap and rub input in a very quantitative manner. Many of the papers we have read through are very qualitative in their analysis of their system/device. It was nice to see these researchers perform two very quantitative studies on top of asking users to evaluate the device qualitatively. One of the big things left unanswered in the paper was where to put this device. And partially because I do research with rings, and partially because it makes perfect sense, I believe getting that additional degree of sensing into a ring would be ideal. The hands and fingers are where we can sense best, and additionally, if we were to place this device on the tip of an unused finger (i.e. maybe the ring finger of the hand holding the device) we could gain a slew of additional feedback information from an otherwise unused finger. Additionally, I really enjoyed that the users were attempting to recreate a human-human interaction thought their entire research project. It seems like modeling an interaction with a computer off of this sort of interaction is the way to go. It feels natural and could potentially be learned much more quickly. TeslaTouch: These researchers present a device that can provide tactile feedback in the form of perceived surface roughness modulation. They achieve this by applying a variable electric field to a screen and essentially varying the frictional force between the finger and the screen. One of the great benefits this system provides to the world of HCI is giving the ability to put this system into currently operational devices, and augmenting the user tactile feedback functionality of the system. The other tactile feedback devices we have read about are either additional devices or components that need to be added on. This setup however truly turns an ordinary touch screen into something that can talk back to the user in a meaningful way. And, it achieves this without needing to distort the screen at all. One of the truly amazing parts about this technology is the lack of moving parts. There is nothing being actuated. The user simply perceives a difference in surface roughness without the surface of the touchscreen changing at all. This system does however leave a bit to be desired. Firstly, the finger must be moving for any additional input information to be conveyed. It works by modulating the friction between the finger and the screen itself by applying a field to the device, so the finger is only going to be able to perceive this change in attraction to the surface of the screen if it is moving. This is not ideal for many situations, although it could be beneficial in some. Additionally, there is no way for different fingers to feel different feedback. This is certainly a huge problem, and one that would definitely need addressing if this were to be turned into a real product. There would need to be a way to modify which parts of the screen the field could propagate though, and thus change the roughness of.

MudPad: This thesis gives a very in depth explanation on tactile feedback and proposes a device that uses a magnetic liquid to change the computing device’s surface properties. These researchers, clearly having spent a long time on this project in that it was thesis work, did a very thorough job in properly prototyping and testing their device so that it would up being highly functional and quit polished. They began with their first prototype which was essentially just a device used to determine if the magnetorheological fluid could be used as a viable option for setting and altering the tactile properties of the surface. Once they determined it could, they made two additional prototypes, finally ending at their proposed system. This device is pretty incredible. It can address 84 separate locations on a 2x2 cm surface area. When the researchers started determining application for this system, the ran user studies to determine what a user could and could not distinguish. It was a bit unfortunate for me that they only used two different viscosities (fluid and stiff) for the magnetorheological fluid. I feel as though this is severely limiting the applications and power of the system. The real money maker in haptic feedback, for me anyway, is making typing on a touchscreen viable. The lack of any meaningful feedback when typing on a smartphone or tablet, is endlessly frustrating, and something that keeps us from using our mobile devices to their full potential. This could wind up giving feedback that is superior to what we feel when typing on a physical keyboard. The paper mentioned vibrating the a key if a word that is not in the dictionary is entered into the system. The user could define his/her own functions to give added information when certain events occur. Maybe a user always types a certain word incorrectly, he/she could set the keyboard to vibrate or modulate itself in a certain way until the user learns not to type that anymore. The user studies show that using MudPad as a number input device is essentially on par with using a normal keypad. They show that it is only a few percent slower. And with all the possible benefits of adding in additional functionality, I think users would be able to use it much more effectively after new interesting features were implemented on it.


David Burnett - 3/11/2013 23:57:58

"TeslaTouch"

TeslaTouch is a haptic touchscreen using electrostatic attraction to dynamically change the coefficient of friction between the screen and the user's finger. Users are given the perception of an otherwise smooth screen becoming rough, without the sensation of surface current or static electricity.

This input method offers a rich system output seldom available in digital systems but always available in the real world. With properly tuned eletrostatic patterns, users can experience new attributes like roughness and inertia which don't exist in the digital world but could go a long way toward bridging the perception gap. Data presented in the paper supports the claim that a variety of input sensations are possible.

Generating an electric field for use with this project is simple, so many types of surfaces can be augmented cheaply and easily. The electronics are solid-state and robust, so this is also suitable in situations where long-term stability is necessary.

The project suffers from a small usability downside in the necessary wrist strap pictured in all experiments. Clever incorporation of a ground strip, or differential voltage between two hands using the screen at the same time, could make using TeslaTouch even more immersive.

Lastly, the paper seems not to mention the bane of all capacitive-based input systems, and perhaps a key downside to this output system: skin conductivity. Perspiration often interferes with electrostatic sensing, and it would be very good to know how this project is experienced by a wide selection of skin types in various relative humidity conditions. "Dry outer skin" is mentioned only in respect to the 1954 Mallinckrodt paper and nowhere else.


"Tapping and Rubbing: exploring new dimensions of tactile feedback with voice coil motors"

This work explores two dimensions of voice coil output for vibrotactile feedback: with a hand resting palm-down on the device, one voice voil moves horizontally to rub a surface along the user's hand, and one moves vertically to tap the palm.

The results of the paper demonstrate a continuum of output. Users are able to discriminate between many different types of each output axis, leading to more dense user feedback than a morse code-like mobile phone buzz.

Unlike some feedback actuation methods, this device can be programmed easily. Voice coils are very well-established technology and the skills to correclty drive them with matched signals are easily obtained. The haptic nature of the output allows for rapid tuning.

Users feedback is difficult to clearly distinguish for future use; some is quantized, but much of the results are qualitative. Furthermore, the specific interpretations for each type of response seems very personal, making it more difficult to build a universally-recognizable system.

Given the size of the device, integration into desktop, mobile, or wearable applications is difficult to picture. This is further complicated by the restricted positioning: the need for an interaction port between the device and the user, which further constrains development.


"MudPad"

This device uses a thin squishy overlay for use with pressure-sensitive screens filled with magnetic fluid that can be controlled with a magnetic array underneath to create distinct sensations depending on array patterns.

The variable density screen surface could allow soft on-screen keyboards to be as intuitive as physical keyboards are today, allowing hands-free operation with minimal drift during use.

It is also one of the only truly passive haptic output systems, capable of presenting data completely distraction free. One must physically interact with the device to, for instance, learn whether new email has arrived.

The paper seems to lack any conclusions about whether the input method was a success. Many potential applications are presented without reporting on implementation, and though some versions of the paper mentioned sample applications, usability was not commented on.

It was difficult to imagine how MudPad would feel when used; probably one if its biggest weaknesses is needing to experience to understand, and thus to design for. The "muddiness" of the device has few analogs in the real world, which increases design difficulty.


Hallvard Traetteberg - 3/12/2013 0:47:42

- Haptic feedback -

Bau, Poupyrev et al, "TeslaTouch"

The paper presents a technique for providing haptic feedback to finger-tips on top of touch displays. The technique utilizes how the friction sensed by our finger-tips can be changed by means of an alternating electric current on a surface. The paper describes how a fairly standard touch screen using capacitative sensing can be modified to generate the electric waves. Waves of varying frequency and amplitude give rise to different sensations, and the paper also describes qualitative and quantitative experiments conducted to understand how users experience the effect. There are mainly two shortcomings (of the technique, not of the work): 1) a finger must be in motion to sense the (change in) friction and 2) the effect is global i.e. cannot be targeted at a specific point or finger. Hence, a user interface utilizing this technique must be designed with this in mind, and the paper provides some suggestions for this.

The work seems well done, by both utilizing a known effect in a novel way, and empirically studying the effect. I'm a bit surprised that they don't suggest applying it for interfaces for the blind, since they already are used to reading with fingers. Perhaps the overall layout of windows and UI elements could be indicated using this effect.

Li, Baudisch, Griswold, Hollan, "Tapping and Rubbing: exploring new dimensions of tactile feedback with voice coil motors"

The paper describes how the coil motor used to move the read/write head of a hard disk can instead be used to provide haptic feedback to a user. They empirically study two kinds of feedback, taps and rubs, that are they argue are particularly relevant since they are used among humans.

I don't find this paper convincing, since I cannot imagine how the feedback apparatus may be easily integrated in laptops and mobile devices. I'm also no convinced by their arguments that taps and rubs are particularly relevant as human-like feedback.

Jansen et al. "MudPad"

The paper describes a technique for providing haptic feedback on a surface, based on changing the viscosity of a sandwiched fluid using an array of coils. The fluid may be made soft or stiff quickly, albeit with a very low resolution. There are lots of limitations, most notably that the required magnetic field is fairly strong, hence each magnet is large and the resolution low. Also, since the magnets are opaque, any content must be projected from above. Still, I find the work fascinating, as the technique can provide local feedback of a certain quality. Unfortunately, the paper does not report on empirical studies on users, nor on how it has been exploited in real applications.


Sean Chen - 3/12/2013 2:33:29

TeslaTouch

TeslaTouch provide users tactile feedback. It uses electrovibration to vary the friction between the touch surface and fingers. It has no moving parts and therefore has low power consumption and lower cost.

Being able to receive haptic feedback is pretty useful sometime, SLAP being one of the examples. The electrovibration seems to be very feasible. The prototype tablet is not too thick (comparing to the MudPad). The paper did some good evaluations. I also like that it has quite a few use cases that seem pretty compelling.

One drawback is that it can only provide one tactile signal to the entire surface. However, only moving fingers can feel the feedback. This mitigates the issue since it makes sense for user to focus on one task/movement at a given time. Having different feedbacks in more than two fingers is not likely to be perceived well by humans.


Tapping and Rubbing

There are some cases where visual and auditory feedbacks are not as appropriate, such as while talking to someone, in a lecture, or driving. Haptic feedback can be helpful for these kinds of situations.

This paper is trying to explore other ways of haptic feedbacks besides vibration. It did quite amount of evaluation. However, it doesn't surprise me that human skins are capable of distinguishing various frequencies and amplitudes. Thus, this result doesn't seem to add much value to the research. I wish it had come up with more concrete use cases.

The paper also compared rubbing with vibration on a mobile phone. But rubbing has its limitations. It can't be implemented on the entire phone, which means user has to hold it in a certain way to get touched with the rubbing part. It also means that putting into ones purse would not work while vibration might still sensible.


MudPad

This paper introduce a design which gives localized active haptic feedback by using magnetorheological fluid and an array of electromagnets to actuate the fluid.

The core advantage of this design is the localized part, which is lacked in TeslaTouch. Although most of the cases users need not have all fingers receiving the feedback - even for virtual keyboard, we mostly tap one key at one given time, this paper did show some more creative use cases such as musical applications where one might be pressing multiple keys at once.

The disadvantages of this design is size. It requires top projection. In addition, the natural characteristic make it hard to become a handheld device due to the thickness. Therefore, it's currently only suitable for tabletops.

Since it's a new way of feedback, I wish the authors had done more evaluation regarding how users feel about the feedback, how distinguishable it is, and so forth.