2003-12-19

On The Phenomenology of Synaethesia


Some people "experience sensations in multiple modalities in response to
stimulation", 'see' printed numbers as different colours (5 as green; 2 as red)
-- synaethesia.

My are my notes on The Phenomenology of Synaethesia, Ramachandran and Hubbard's
summary paper of their research.

http://www.imprint.co.uk/pdf/R_H-follow-up.pdf -- Phenomenology of Synaesthesia
http://www.psychiatry.cam.ac.uk/isa/whatis.html -- Good summary of synaesthesia
http://dalston.ku24.com/cluster/archives/000280.html -- link to paper

(when I think of it, it's a bit like code highlighting in BBEdit.)

#

There are some fantastic findings, like whether the font of the number changes
the colour sensation: a font like Gothic may sometimes evoke a stronger colour:
"We suggest that such fonts might serve as 'hyper-normal' stimuli that evoke
even larger responses from the grapheme neurons than a more 'prototypical' font
might." (Which, from an interface perspective, is wonderful: that the brain has
a shape that binds around sensations, and this shape itself can be gamed in
other ways.)

#

Looking for correspondances across synaesthesiacs: "For instance, it may not
initially be obvious why the arbitrary vowel sequence AEIOU should map in a
non-arbitrary manner to a certain sequence of colours (and we have seen some
hints of this happening). But the point is that the sequence AEIOU may not be
arbitrary. It may reflect progressively anterior mouth and tongue articulations
which in turn might be mapped in a topographically organized phoneme space."

(At which point I have to mention both the Great Vowel Shift (15th-18th century)
and the more recent Northern Cities Shift: a vowel-sound moves to disambiguate
from another, this causes a knock-on effect; the consequence is that internally
to a dialect people can understand each other better, but between dialects
(between cities in North America) people have trouble comprehending one another.
I believe bread-related products also underwent a similar shift moving across
the Atlantic: each of biscuit/cookie/rusk/scone/bun/cake rotates along the
axis.)

http://alpha.furman.edu/~mmenzer/gvs/what.htm -- Great Vowel Shift
http://www.ic.arizona.edu/~lsp/Northeast/ncshift/ncshift.html -- Northern Cities
http://interconnected.org/home/mini/archive/2003/03/16/ -- more ling. links

#

A clever-but-simple experiment showing that synaesthesia does indeed occur on
the sensory level (it's not imagination or just a strong memory association):

"""
For example, if several 2s are scattered among a matrix of randomly placed 5s,
the global shape formed by the embedded 2s is very hard to discern; normal
subjects take several seconds to find the shape because the 2s and 5s are
composed of the same features three horizontal lines and 2 vertical lines. But
if a grapheme-colour synaesthete looks at it he instantly Ñ or very quickly Ñ
sees the global shape as a red triangle or square against a background of green
2s.
"""

(So if comprehension == navigation, is synaesthesia a good metaphor for gaining
expectations of a system? Once you understand how a scientific paper is
organised (its semantic shape) then you don't just read the paper, you 'see' its
shape, concentrate on divergences, etc. It's an unrelated symbol that's fired in
addition to just your reading symbols: a multimodal perception.)

#

And linking the asymmetry of synaesthesia (graphemes to colour but not vice
versa) with the nonarbitrariness of metaphor ("For example, you say 'loud shirt'
but you rarely say 'red sound'; you say 'sharp taste' but rarely 'bitter
touch'.") -- is this to do with evolution and neural constraints?
Correspondingly we share facial expressions from being infants and smells are
talked of as tastes; are these too metaphors? "There are also hints that
patients with right hemisphere lesions show problems with metaphor. It is
possible that their deficits are mainly with spatial metaphors, such as 'He
stepped down as director'."

(And then later, "It may be because puns are in some ways the opposite of
metaphors; a metaphor reveals a deep similarity whereas a pun is a superficial
similarity masquerading a deep one, hence its comic appeal.")

#

This is startling:

"""
We offer an empirical solution to the question that has long puzzled
philosophers: How does the activity of neurons in the sensory (and other areas)
in the brain give rise to the subjective quality Ñ the 'qualia' of sensations
like red or green or pain? The neurons in V4 (and other colour centres) are not
that different physically from (say) the neurons in auditory cortex concerned
with hearing, so why does their activity feel so utterly different? One tool for
probing this question is the 'martian colour effect', an unusual tinting of
colours evoked synaesthetically, which is most obvious and pronounced in a
colour-blind synaesthete we have tested, but occurs in 'regular' synaesthetes as
well. We attribute this colour distortion to the fact that the colours evoked by
cross-activation in the fusiform 'bypass' earlier stages of colour processing,
and therefore may confer an unusual ('martian') tint to the colours evoked. This
is an important finding, for it suggests that the qualia label Ñthe subjective
experience of the colour sensation Ñ depends not merely on the final stages of
processing but on the total pattern of neural activity, including the earlier
stages.
"""

(which only goes to show that models aren't the system, and that a model will
always have (a) parts of the system they can't imitate, and (b) the model will
be able to imitate [things? what? qualia?] that the system can't. Synaesthesiacs
can perceive colours they can't sense!)

#

Imagined grapheme-colours are brighter:

"""
The reverse side of the coin from being 'blind' to a character is to imagine a
number or letter that is not physically presented. Many synaesthetes reported
that when they visualised a number in front of them it was, surprisingly, more
strongly coloured than when they looked at a real number. We suggest this is
because when you imagine something visually there is activation of the same
brain areas Ñ such as the grapheme area in the fusiform Ñ that are driven by
actual physical colours, but without the real black (or white) number coming in
from the retina and competing with the colour experience, this imagined number
actually evokes a stronger experience of the corresponding colour.
"""

#

So synaesthesia is hardwired metaphor, and who knows why it's there, but means
we must have pre-determined strong associations between certain symbols already.
What do we always use space for, what is it a strong-metaphor for? What does
happen if that lacks? Can anything like this happen in reverse (two modalities
combining, what's the effect of breaking an expectation?)?

#

This is an interesting form of exaptative feature: we develop a particular way
to relate to space, say, a series of expectances and the-way-the-world-works,
then we notice that another system works in a same way and apply the same model
to it, and that becomes hardwired in the brain. So we can treat the world of
ideas like space, and abstraction like vision (seeing only surface features).