Easy sensory augmentation?

For ages I’ve been toying around with the idea of building an ear ring with a vibrating servo that I could hook up to a sensor for UV light, carbon monoxide or similar. What’s held me back is mostly not having the skills needed and too little time to start learning from scratch just for this.

However, I came across with a company that sells vibrating wrist bands that connect to mobile phones. Now I’m thinking this might be a super easy way to try adding sensory input for something interesting. Have my phone relay some cool information to the wrist band and see how that turns out.

For what info to receive, some things that have crossed my mind so far are:

  • rain forecast
  • wifi
  • lightning strikes

I don’t like how this, even if it works well, would be totally dependent on my phone and probably internet connection too. It seems really easy to pull off though, and I love the idea of feeling how a thunderstorm is moving in from beyond the visual range…

2 Likes

welcome to the forum!

So something kinda like Neil Harbisson but maybe without the skull screws?

link?

So I guess the question is, what do you want to focus on?

hook up to a sensor for UV light, carbon monoxide or similar

or

have my phone relay some cool information to the wrist band

… the problems and challenges with doing on-body sensor conversion - converting one sense (UV light) into another (tactile vibrations at the wrist) - can be many. The title of your post includes the word “easy”, so it seems the phone / wristband idea is a very good place to start, then go from there.

4 Likes

Thanks.
Yeah, I can see the comparison with Neil Harbisson for sure!

Here’s a link to the wristband.

I’m quite sold on trying the wristband and phone combo. Piercing would have been cool - it’s not bone screws :slight_smile: but still feels a bit less like something you only wear, and losing a sense because my internet goes down doesn’t seem quite right. But the wrist band seems like the way to stop dreaming and actually try something out.

I’d be happy to hear any ideas on what to receive as tactile input.

There are a bunch of real time meteorological measurements on the internet that can be localized, so I feel those are interesting. Other accessible real time data… Movements of a stock index?

I guess one could do something with the GPS, like feeling the movement in the cardinal directions. Or the distance from the nearest… body of water? underground station? something?

It’s quite a shift, from my earlier idea of actual sensors, to anything I can pull off the internet. I’m still trying to cauge the possibilities.

Ah interesting… so it’s designed for blind people. I can see though how it might be useful for everyone in certain contexts.

If a safe implantable battery was possible, one of the things I’ve always thought about doing was creating a special “buzzer” that only buzzed for certain phone notifications from special people… or maybe just one person. A lot of people be like "hey just turn on Do Not Disturb mode but then make exceptions for specific people and notifications, but that never works right… there are too many layers… like if DnD is on but I have exceptions, but then my ringer volume is down or silenced then I don’t get the notification anyway… I want a way to put my phone in my pocket with everything silenced, yet have a very direct and reliable way of being notified that specific people are trying to reach me, regardless of my sound settings.

This band might be work experimenting with… if it had an open API to work with.

Anything you can quantize, can be learned as a new “psychic” interface/language, if contextual and relevant, continual.

From a phone, there’s a lot of information that can be useful, but may not class as a direct sensory stream, for forming deeper connections conceptualy/perceptually, anyway. Such as weather, that’s less often feedback->direct experience feedback than continual CO2/VOC level feedback. Lots of wiggle room here, though - having proprioceptive movement tied in can better help tune in a new sense, I think. Not that this is impossible on phone based signals, or that it’s needed for adding in new senses.
Something as simple as a clock for sense of time, could be turned haptic and fed over a phone, for example. haptic timekeeping device, but just quantize the data differently.

There’s a lot of ideas that work well as senses via a phone link, and others which the delay on the information throughput won’t work as well for, in terms of perceptual/conceptual binding. Compass feedback, gps guidance as you mentioned, for example - good use of phone data!
For other senses, too much delay, and things become jittery, not as easy to wrap up into a perceptual manifold, in terms of symbols.

There’s unfortunately not a lot of good wristbands for doing heavier lifting augmentive tasks, although people also haven’t really pushed what’s possible just via a single haptic feedback point and quantizing information.
You may be able to find a used neosensory wristband on eBay, although those cost quite a bit still, all things considered. Maybe a bhaptics wristband, although you’d have to reverse the protocol sending side (not impossible, sense shift firmware for esp32 does it the other way, emulating their hardware, so may have some good starting points)

However, you can get 8x output (or more) haptic wristbands if you DIY it for sub $50, if crafty.

I’ve been wearing some custom haptic wristbands daily for the past couple years to do extra thermal perception - Sensory Weavers. Like how a telescope or microscope gives lens into new information, these sorts of technologies (from me and others), are sorts of a psychoscope.

So when saying anything you can make a reliable sensory thread for, I do really think there is so much more we can weave into cognition, in theory.
Someday I’ll have DIY parts/kits for the ones I make - but it’s actually not that hard to get into, as far as DIY projects go, even with all of the shelf parts.
The software stack is the more complex part, which even someday that should be as easy as getting into custom 3d printing.

There’s so many possibilities of perspectives to be taken. If you’ve any questions towards specific implementations feel free to ask - lots of random information, references floating around in the head.

1 Like

Thanks for replying @curiosiate. It seems you’ve got experience in just the experiment I am considering, that’s great! I went and read some of the texts you have online, it’s interesting stuff.

Can you explain how you mean proprioceptive movement to help in tuning in a sense?

I’m wondering about the suitability of the sort of data that is not often present. Like I love the idea of lightning strike sense, but because of seasons, there might be months at a time without any where I live. So maybe that wouldn’t work that well. But with such a huge set of potential data sources, I’m thinking there must be some that would be awesome.

Proprioceptive movement helps utilize the place/grid cells more, to associate via multiple means, a new sensory modality. Bimodal stimulation/association, having in/out control over the sense and feeling how those movements modulate the information in relation to other senses.

There are different sorts of feedback that can come from haptics.

Something like lightning strikes would probably come across more as an alert than an innate sensory experience, but exploring a historical dataset of storms, and feeling via navigating the data how it shifts and adapts, may give that sensory/conceptual link over enough usage.

So weather info may be more like someone whispering in the ear, than you conceptually seeing as a modality and means to associate information and connections together over that comes from sensory weaving.

Something continual, and contextually meaningful, will map to lower levels in the mind, allow more resources conceptual/perceptual side to “grow” into it - and may not suffer as much as the other more alert-based method might for things like noisy environments shifting attention away from tactile awareness.

Not saying proprioception is needed, just it can help - continual feedback can still be useful (like the timekeeping!). More points of cross reference, more seeds and roots the mind can use to push down deeper, be it continual feedback, or contextual meaning.

There is a whole game and balance to finding the right mix of both, I think.

Lots and lots of potential data sources to thread into the tapestry of mind.

Hello,

I am currently building a sensory augmentation device with haptic feedback using piezo actuators. The plan is to make a modular system which is controlled over a phone app. Data could be sent through the phone or from added external sensor. Or even a mix of these if needed. I am looking for test users for this once the next iteration of the device is ready. (Hopefully in October). @unihiutale, based on your nickname we might be around the same region - I am working in Helsinki. Maybe you would like to be one of the testers?

This project started as an artistic exploration to the sensory augmentation topic. In fact, the first device is on display until the end of August at Mänttä Art Festival. The piece displayed there is a necklace that chokes the wearer based on the surrounding co2 level. It works with air pressure created with a miniature piezo pump (limited to 40 mbar for safety).

I have also made a “compass belt” for myself where vibration tells the North. While developing the compass and co2 senses, I realized it is quite a bit of work to create one of these sensory devices, so I am now focusing more on the modularity to make a device/platform where different input data/ different senses could be easily tried out.

PS. Very interesting work with sensory weaving @curiosiate and nicely documented as well!

3 Likes

Oh that’s very cool! Is the plan for this system to be open source?

Love the idea of non-vibration based feedback, in the case of the air pump - compass belts are amazing too. These sorts of things very much are artistic explorations, I think - at least if not wanting to fall foul of health/safety standards for devices claiming to “solve” or “cure” issues, even if making auditory→haptics or otherwise may help people out in theory still. Can’t market it as such, sadly - at least in some countries.

Do you happen to have links to these projects somewhere? Would love to add them to the wiki page I’m making to catalog different builds I’ve run across for devices that do sensory weaving stuff (trying to use it as a more generic term for any devices that do sensory substitution, expansion/addition)

You are indeed right on the hassle to get going - that’s why I’ve also been making stuff towards that goal of making a modular system.
Currently have the network stack/variable management done for C++ and javascript, along with a modular control panel - but phone apps I’m less working towards versus web page based control for now (although ESP32 chips can host that, so it kinda works like that, and maybe web bluetooth API works on mobile? )

Long term goal is something I’m calling a “manifold synthesizer” - which is where I’m still working on the backend firmware architecture to enable that, along with elf-loading for sideloading apps.
First is moving to a freeRTOS back end versus the generic Arduino core, which while I’ve made those libraries thread safe already, the libraries for sensors will need to be looked into unfortunately. :frowning:

If someone knows programming though, the older test setup I’m hoping to clean up and open source before the MIT augmentation summit this year, should help with that sort of testing/trying out by that variable/method exposing and saving. (if I can do it in time, have to do some mechanic work on my cars too so we shall see)

The more ways people can explore senses, for different use cases, the better I say! Freedom of perception is needed! (or as I’ve written a bit on, the right to cognitive lightcone expansion)

So whenever you do make your system more public, would love to see it and share it with the places I’ve been talking to people about the sensory weaving stuff, to help individuals have many paths towards that (if open source, but even if not they would think it cool - if you have discord, PM me and I can send you some invite links if interested!)

1 Like

Yes it will be open source, eventually.

For now, there is no proper documentation yet of the existing pieces, as the compass belt was only out of my own curiosity for personal use, and the co2 choker is still on display at the art exhibition. There are some images at my latest Instagram post though @pekkovasantola.

With the new modular system I haven’t yet got around to the software side much, as I’ve focused on the piezo actuators and playing with the possibilities of different waveforms. But the plan is to work on this a lot in the following months. Maybe we can join forces and develop the software together!