MUSIC FOR THE DEAF (copied, without the background music, from http://neuraleffects.com ) Having checked out and improved some of the ideas expressed below, I've started working on the development of music for people with cochlear implants. Deaf "listeners" will be able to appreciate subliminal effects, such as evoked phantom extra ears (created by structuring the form of the sound information which the implant sends to the brain), which are not available to advertisers and composers of sound effects and other music created for people who can hear (our equivalent to "phantom ears" is probably the "phantom voices" that some commercial adverts produce). The "phantom ears" effect enables the implant recipient to hear and understand more than one strand of information at a time. Signals from the cochlear implant corresponding to sounds beyond the normal frequency range for hearing people are also expected to be useful in such music. It is not possible for people with normal hearing to perceive effects such as "phantom ears" properly, either through loudspeakers or headphones, because our ears are a physical barrier which prevents any sort of direct 'programming' of the auditory nerve as though it were a data bus in a computer, but a cochlear implant completely by-passes the (usually profoundly deaf) ear's natural workings and defenses. The music being developed consists of data files containing patterns in the same format as the usual cochlear implant control signals sent to the inner ear, to be listened to by people who are already fitted with re-programmable implants. In due course, I expect to present some of my compositions to manufacturers of cochlear implants, in the hope that the current apparently 'stuck' situation in the market for cochlear implants and similar products can be alleviated: at present, no major improvements in hearing technology are expected other than in biological regrowth of the inner ear hair-cells, and manufacturers are unwilling to admit that the design of implants already in use is based on an out-of-date and incorrect model of hearing (the "Place Theory"). Cochlear implants should be the technology of choice for people who want to directly interface their brains with computers, so that they can use high-speed learning tools and similar products. Deaf people who already have programmable cochlear implants may be expected to earn income as co-operating experimentees during product development within the industry. Phantom ears The term 'phantom' is also used by doctors in a medical context when referring to phantom limbs and similar, and may have some bearing on the effect I am describing: It is possible to multiplex the input data to the ear and send two or more separate sound streams, using a join technique which helps the brain pattern-match the signals and identify it as two separate things. The data is the cochlear firings relevant to the pressure differential information that programmable cilia would respond to. At any 'splice-point', the normal ear would hold the information equivalent of a rolling average of the most recent sound (the last 30th of a second or so), contained within the cilia which are recharging, and is a low-quality delayed form of the information which has just been sent along the auditory nerve. In a cochlear implant, the expected rolling average can be replaced with a different rolling average at the splice point (one generated from a different sound). Using this splicing technique every 40th of a second or so, two separate sound streams can be welded together in a way that the brain does not usually ever hear (except perhaps in dreams). When hearing the multiplexed sound, the brain has some options: it can understand both sounds separately with no problems (as easily as it already can with two normal ears); it can understand the sound as a mixture of the two; or it can understand one properly and mask the other. "Phantom ears" can presumably be evoked in deaf people in much the same way that "phantom voice" effects are already produced, from the brains of normal listeners, by sound engineers working in the recording industry. from Bekesy to Cochlear Implants Because of the work of Bekesy and others, it is known that the inner ear responds to different frequencies at different places along the length of the basilar membrane, by sending electrical impulses along the auditory nerve into the brain. This principle is at the basis of 'cochlear implant' technology, which is used to help deaf people in situations where the ear is severely damaged but the auditory nerve is unimpaired. A cochlear implant is in effect an 'artificial ear', which replaces the functionality of most of the ear and the cochlea: a microphone earpiece transforms sounds into electrical signals corresponding to the frequency bandwidths of the 'formants' of human speech, and a thin wire inserted into the cochlea stimulates the basilar membrane at places along its length corresponding to these frequency bandwidths. For some reason, the performance of cochlear implants has been disappointing. Recipients often report that the implants only help them distinguish between sounds and silences - if the implants are 're-tuned', sounds seem higher or lower for a while, but the effect soon wears off. Although recipients are encouraged to persevere, on the principle that their brain would gradually learn to make sense of the input noise, many of them prefer to leave their implants switched off most of the time because they are too noisy. One could suggest that these patients were unlucky - perhaps their auditory nerve was too damaged for cochlear implants to work for them. However, one rarely reads believable reports of lucky successes in cochlear implant operations, where patients who were previously deaf have become able to hear speech relatively clearly - many reasonably intelligent adults who have become profoundly deaf can lip-read and speak with remarkable accuracy even without cochlear implants, and one would expect genuine improvements to be more widely-known. The popularity of cochlear implants these days may be related to the success of laser surgery for cataracts and other almost miraculous and relatively cheap 'cures for blindness', unfortunately, the equivalent technology for the ears provides none of the hoped-for miracles and cochlear implants have already managed to acquire a bad name for themselves amongst the deaf community. The idea behind cochlear implants is correct (from Bekesy), today's technology is certainly capable of the speed and precision necessary to transform sounds into the required electrical signals (a normal cochlea sends only a few thousand signals per second along the auditory nerve), and technicians can tune the frequency response from different places along the cochlear implant to a very high accuracy for any particular recipient - sufficient that one might expect the technology to be not only adequate for human speech (which is the focus of cochlear implant research these days), but also for sounds as varied as thunder, Beethoven symphonies, and birdsong. So why aren't cochlear implants more successful? The answer must be that the 'place theory' model of hearing is incomplete. My interest in the subject came about as a composer of music, with normal hearing. I was trying to construct a filter which would stimulate the inner ear in patterns which are unlikely to occur in nature, in the hope that interesting and unusual sounds would result, and I developed a computational model of the ear which I think can be applied in the area of cochlear implants for deaf people, which unlike the 'place theory', has yet to be falsified. The model is as follows: Sounds are caused solely by variations in air pressure over a period of time, and computationally, the ear/brain connection is modelled as a large number of pressure-sensitive synapses, equivalent to the cilia inside the cochlea. These synapses are like miniature batteries, which are charged up by either a decrease or increase in pressure (on the outer hair cells of the inner ear, the tiny stereocilia which happen to be facing any oncoming pressure increase bend backwards because of their aerodynamic shape, charging up a contributory amount of energy to the OHC synaptic "battery" - the stereocilia on the same OHC facing in the opposite direction would respond similarly to a decrease in pressure). When a battery is fully charged, it discharges its contents (fires) onto the auditory nerve, and then starts to recharge according to the subsequent sound pressure variations. The sense of hearing comes from the brain's statistical interpretation of the firing patterns sent from the cilia. The synapses behave like thousands of individual primitive ears, each sending their own coarse interpretation of the sound pressure variations along the auditory nerve and into the brain, and together they form electrical impulse patterns which the brain attempts to interpret. It is important to realise that these patterns contain temporal information because of the time between firings of different synapses, so the brain can deduce frequency information from the distance between the electrical impulses corresponding to these firings. Amplitude information is deduced from the quantity of firings occurring for a particular pattern. While the impulse patterns are travelling in the brain, they are matched against patterns from previously-heard sounds which have been stored in the brain's auditory memory. During this process, neural pathways in the memory are strengthened, resulting in the brain deciding on what the sound 'is'. I could go into considerably more detail, but I assume that anyone who is familiar with the problem will understand what I am getting at, and will be able to adapt or add to the model where required. Allow me to put the problem into a perspective. The fundamental idea behind cochlear implants, that different parts of the cochlea are sensitive to different frequencies, cannot be ignored. Direct readings of the electrical output from different parts of the basilar membrane, when stimulated by sinewaves of different frequencies, show that for each frequency, there is a maximum output at a corresponding part of the basilar membrane. Unfortunately, there is these days no credible 'causal explanation' for the above fact. Bekesy (1961) gave an explanation, based on vibrations of the basilar membrane, which satisfied scientists of his time. The reason for Bekesy's basilar membrane conjecture, was that individual cilia are so simple biologically that it is just not possible that they could be some sort of 'sound frequency' detectors, which send a signal to the brain whenever they detect a particular sinewave frequency from the pressure variations in the cochlear fluid. If individual cilia could achieve such a miraculous feat, why would nature ever have needed to evolve intelligent brains at all! Since then, the mathematics behind Bekesy's explanation was discovered to be too simplistic in the area of 'frequency dispersion', and Lighthill et al. produced an improved mathematical model which covered the fault. More recently however, it was discovered that the cochleas of patients whose basilar membranes had been 'solidified' by a fungal disease, so they could not be vibrating, still produced the frequency dispersion characteristics of the Bekesy/Lighthill model. From this it has been deduced that movements of the basilar membrane are irrelevant, and that no matter how unlikely it might seem, the hairlike cilia must be the elements in the ear which detect frequencies, as they are the only other candidate for frequency detection within the ear. When experimenters test the cochlear response to validate the 'place theory' (that each cilia responds to a specific frequency), they input a pure sinewave of a particular frequency to the ear, and then observe the place in the cochlea which gives the maximum output. My model explains the place theory observations as a resonance effect caused by the experiment. One of the more surprising things about the ear is the low amount of electrical impulse information which it sends along the auditory nerve, compared to the 40,000 16-bit words per second required for digital audio. The several thousand cilia each have a maximum repeat firing rate of between 50 and 150ms, so that during loud sounds most of them are incapable of firing more than half the time because they are 'recharging their batteries', and typically only a few thousand impulse signals (all pretty much the same magnitude) are sent per second into the brain.> maestro schrieb: > > A cochlear implant is in effect an 'artificial ear', which > > replaces the functionality of most of the ear and the cochlea: a > > microphone earpiece transforms sounds into electrical signals > > corresponding to the frequency bandwidths of the 'formants' of human > > speech, and a thin wire inserted into the cochlea stimulates the > > basilar membrane at places along its length corresponding to these > > frequency bandwidths.> Ein mikromechanisches Cochlea-Implantat: > http://physicsweb.org/articles/news/9/1/12/1 > http://www.eurekalert.org/pub_releases/2005-11/nsf-nsb110305.php > http://www.newswise.com/articles/view/515884/> > Aktive Schallverst=E4rkung im Ohr: > http://www.wissenschaft.de/wissen/news/247808.html > http://www.uni-protokolle.de/nachrichten/id/20115/ > http://www.sciencenewsdaily.org/story-8322.html> Signal-Pfade beim H=F6ren: > http://www.maxplanck.de/english/illustrationsDocumentation/documentat... > http://www.eurekalert.org/pub_releases/2002-05/jhmi-hsr050202.php > http://www.innovations-report.de/html/berichte/biowissenschaften_chem... > http://www.eurekalert.org/pub_releases/2005-05/nu-pef052505.php > http://www.sciam.com/article.cfm?articleID=3D000913B8-FBB8-1C5A-B882809... > http://www.eurekalert.org/pub_releases/2006-02/uorm-hlt021006.php > http://scienceblog.com/cms/node/4361 > http://www.wissenschaft.de/sixcms/detail.php?id=3D148846 > http://www.netzeitung.de/wissenschaft/309160.html > http://biology.plosjournals.org/perlserv/?request=3Dget-document&doi=3D10=.=2E.> http://www.sciencedaily.com/releases/2002/12/021219065710.htm> Haarzellen z=FCchten: > http://www.newscientist.com/article.ns?id=3Ddn4311 > http://science.orf.at/science/news/132660 > http://www.eurekalert.org/pub_releases/2003-05/uomh-gtg052703.php > http://www.netzeitung.de/wissenschaft/325558.html > http://www.wissenschaft.de/wissen/news/248156.html> Sinneszellen, Stereocilia: > http://www.wissenschaft.de/sixcms/detail.php?id=3D149472> Ohr-Evolution: > http://www.wissenschaft.de/wissen/news/249112.html> Sonstiges > http://www.heise.de/newsticker/meldung/56186 > http://idw-online.de/pages/de/news45800Perhaps I should explain better. I am suggesting that in the design of cochlear implants, one can ignore frequency and amplitude considerations. Given that the travelling wave theory is known to be incorrect, and that therefore synapse firings associated with cilia alone must be encoding the information contained in the stream of rapidly varying pressures presented to the ear, the suggestion is that these synapses behave like miniature batteries whose electrical potential increases when pressure is applied, and they discharge an impulse to the brain when a sufficient amount of the varying pressure has been applied over a period of time not less than their maximum repeat firing rate, which (researchers have shown) varies between about 1/40th of a second for the cilia with the fastest response, and 1/12th of a second for the slowest. Cochlear implants can be programmed with a model which simulates the behaviour of several thousand of these pressure-sensitive 'synapses' when given a typical sound signal. The electric signal which attempts to cause the firings equivalent to the those predicted by the model should be fed directly to all parts of the cochlea. The implant should not perform formant analysis on the input sound and only stimulate (ie trigger) those parts of the cochlea which correspond to the 'place theory' positions of the frequency bands corresponding to the formants, because that does not use the cochlea as efficiently as in normal hearing. If the formant-based (speech-sensitised) programming of cochlear implants worked as advertised, the ordinary sounds of birds, traffic and the weather would tend to sound like a constant stream of voices, and complaints similar to the 'hearing voices' symptoms of schizophrenia would be expected from deaf people with implants. And why have the manufacturers given so little thought to the appreciation of music and natural sounds other than speech? Perhaps the reason that the suggested method is not already used is simply that manufacturing technology is not yet capable of the miniaturisation and precision necessary to electrically stimulate individual nerve cells - there are thousands of cilia but only 22 channels in most cochlear implants. However, it is not known that as many as 22 channels are necessary and it may be that one channel for each ear is sufficient, if the cochlear implant can behave like a single perfect cilium, or as cilia with a perfectly even response throughout the ear, by sending a uniquely noise-free signal (the same from all parts of the implant, instead of the statistically-interpreted fuzzy signal from the ears that the brain usually receives) along the auditory nerve to the brain. These one-channel cochlear implants should also be useful for the partially deaf, if they can replace damaged parts of the cochlea without sending a signal to the brain which conflicts with the information from any remaining working parts of the inner ear (as the technology used in present-day 22-channel cochlear implants would). A typical manufacturer's description of the workings of the ear and cochlear implants can be found at: http://www.science.org.au/nova/029/029key.htm Bekesy's "Place Theory" is explained in laymen's terms, and the website declares that our ears are natural Fourier analysers and that the cilia are like notes on a piano. Continuing their analogy, my own explanation for the Place Theory observations is that they are measuring the vibrational effects of hitting the wooden casing of a piano with a drumroll at various frequencies, and they only give indirect information about the workings of the vibrating strings contained within the body of the instrument. Most laymen are unaware that the physics of sound production from a piano is itself a subject of much debate and disagreement amongst professionals, and there is certainly room for improvement in the technology. [ http://ultrapiano.com ]
phantom ears
Started by ●October 26, 2006