WEBVTT

00:00:00.000 --> 00:00:01.070 align:middle line:90%


00:00:01.070 --> 00:00:03.880 align:middle line:84%
So far, we have looked at the
differences between motion,

00:00:03.880 --> 00:00:05.710 align:middle line:90%
action, and gesture.

00:00:05.710 --> 00:00:09.330 align:middle line:84%
But this concept of gesture
is quite tricky to understand.

00:00:09.330 --> 00:00:12.150 align:middle line:84%
So now we're going to
visit a colleague of mine

00:00:12.150 --> 00:00:15.450 align:middle line:84%
and hear how he is thinking
about this concept of gesture.

00:00:15.450 --> 00:00:15.950 align:middle line:90%
Come.

00:00:15.950 --> 00:00:22.086 align:middle line:90%


00:00:22.086 --> 00:00:23.040 align:middle line:90%
Hello, hello.

00:00:23.040 --> 00:00:23.540 align:middle line:90%
Hi.

00:00:23.540 --> 00:00:24.040 align:middle line:90%
Hi.

00:00:24.040 --> 00:00:28.070 align:middle line:90%


00:00:28.070 --> 00:00:33.020 align:middle line:84%
The baseline of all this is
that music is a multimodal art.

00:00:33.020 --> 00:00:38.060 align:middle line:84%
It involves both the sense of
motion and the sense of sound.

00:00:38.060 --> 00:00:41.235 align:middle line:84%
And, of course, it can involve
also a sense of vision,

00:00:41.235 --> 00:00:44.280 align:middle line:90%
even maybe the sense of smell.

00:00:44.280 --> 00:00:46.020 align:middle line:90%
It's a multimodal art.

00:00:46.020 --> 00:00:50.770 align:middle line:84%
And for that reason, it's
better to try to be precise

00:00:50.770 --> 00:00:52.130 align:middle line:90%
when we speak about it.

00:00:52.130 --> 00:00:55.710 align:middle line:84%
When we speak about
music and body motion,

00:00:55.710 --> 00:00:58.850 align:middle line:84%
we can say we have body
motion to make sound.

00:00:58.850 --> 00:01:02.240 align:middle line:84%
We have body motion
to modify sound.

00:01:02.240 --> 00:01:05.260 align:middle line:84%
We have body motion
that somehow complements

00:01:05.260 --> 00:01:08.250 align:middle line:84%
the sound, like in
dance choreography,

00:01:08.250 --> 00:01:09.670 align:middle line:90%
and so on and so forth.

00:01:09.670 --> 00:01:14.570 align:middle line:84%
So it's clear that at
the end of the day,

00:01:14.570 --> 00:01:18.355 align:middle line:84%
the essential point is
precisely to understand

00:01:18.355 --> 00:01:23.230 align:middle line:84%
how music is composed
of both body motion-- I

00:01:23.230 --> 00:01:26.310 align:middle line:90%
prefer that term-- and sound.

00:01:26.310 --> 00:01:30.730 align:middle line:84%
We can now say we have a
basic understanding of music

00:01:30.730 --> 00:01:33.275 align:middle line:84%
as a phenomenon,
well-established

00:01:33.275 --> 00:01:39.125 align:middle line:84%
both from our own research
and from neuroscience.

00:01:39.125 --> 00:01:43.480 align:middle line:84%
that there is a very
strong hardwired coupling,

00:01:43.480 --> 00:01:47.120 align:middle line:84%
as they say in neuroscience,
between what we hear

00:01:47.120 --> 00:01:49.290 align:middle line:90%
and our sense of body motion.

00:01:49.290 --> 00:01:52.810 align:middle line:84%
And as we know from our
research observing musicians

00:01:52.810 --> 00:01:58.030 align:middle line:84%
and dancers, most people
tend to spontaneously

00:01:58.030 --> 00:02:02.410 align:middle line:84%
associate musical sound with
some kind of body motion.

00:02:02.410 --> 00:02:06.530 align:middle line:84%
Another topic that is quite
important in your research,

00:02:06.530 --> 00:02:09.860 align:middle line:84%
I know, is that
of coarticulation.

00:02:09.860 --> 00:02:11.420 align:middle line:84%
It's a difficult
word, but can you

00:02:11.420 --> 00:02:14.170 align:middle line:84%
try to briefly explain
how you're thinking

00:02:14.170 --> 00:02:15.500 align:middle line:90%
about coarticulation in music?

00:02:15.500 --> 00:02:16.040 align:middle line:90%
Oh, yes.

00:02:16.040 --> 00:02:18.090 align:middle line:90%
Yes, yes, with pleasure.

00:02:18.090 --> 00:02:22.400 align:middle line:84%
Coarticulation means
that in human motion,

00:02:22.400 --> 00:02:24.460 align:middle line:84%
and by the way, also
in robotics, and also

00:02:24.460 --> 00:02:30.610 align:middle line:84%
in animation, you have the fact
that body parts are constantly

00:02:30.610 --> 00:02:31.900 align:middle line:90%
on the move.

00:02:31.900 --> 00:02:36.330 align:middle line:84%
So the easiest explanation
is to look at your mouth

00:02:36.330 --> 00:02:38.600 align:middle line:90%
when you are speaking.

00:02:38.600 --> 00:02:40.510 align:middle line:84%
Whenever you're
pronouncing a word,

00:02:40.510 --> 00:02:42.720 align:middle line:84%
you can see the shape
of the mouth, the lips,

00:02:42.720 --> 00:02:43.640 align:middle line:90%
and the tongue.

00:02:43.640 --> 00:02:45.830 align:middle line:84%
You follow the
motion of the tongue.

00:02:45.830 --> 00:02:49.640 align:middle line:84%
And you'll discover that when
you are saying something,

00:02:49.640 --> 00:02:51.880 align:middle line:84%
you're also preparing
the next sound

00:02:51.880 --> 00:02:53.500 align:middle line:90%
that you are going to make.

00:02:53.500 --> 00:02:58.450 align:middle line:84%
And also, when you're saying
something, you are, in a sense,

00:02:58.450 --> 00:03:01.020 align:middle line:84%
conditioned by
what you just did.

00:03:01.020 --> 00:03:03.590 align:middle line:84%
In music, if you are
going to play the piano

00:03:03.590 --> 00:03:06.190 align:middle line:84%
and hit a key way
up on the keyboard,

00:03:06.190 --> 00:03:09.720 align:middle line:84%
you necessarily have to move
your hand in order to hit it,

00:03:09.720 --> 00:03:13.230 align:middle line:84%
because you don't have a
finger that's that long.

00:03:13.230 --> 00:03:19.270 align:middle line:84%
So this means that you always
are in a context of motion.

00:03:19.270 --> 00:03:22.980 align:middle line:84%
And this is quite
determining for how

00:03:22.980 --> 00:03:27.100 align:middle line:84%
music is shaped, both vocal
music and instrumental music.

00:03:27.100 --> 00:03:31.520 align:middle line:84%
So if you look at
the score, what

00:03:31.520 --> 00:03:34.215 align:middle line:84%
is called Western
common music notations,

00:03:34.215 --> 00:03:38.480 align:middle line:84%
you have dots, C-sharp,
F-sharp, G, and so on,

00:03:38.480 --> 00:03:39.810 align:middle line:90%
which are discrete events.

00:03:39.810 --> 00:03:41.970 align:middle line:84%
But when it comes
to performance,

00:03:41.970 --> 00:03:44.750 align:middle line:84%
the body has to somehow
move between the keys.

00:03:44.750 --> 00:03:48.950 align:middle line:84%
Or in vocal performance, you
have to move from one pitch

00:03:48.950 --> 00:03:50.030 align:middle line:90%
to another.

00:03:50.030 --> 00:03:54.080 align:middle line:84%
So essentially, you
always have a smearing.

00:03:54.080 --> 00:03:57.740 align:middle line:84%
And that's the word I use,
meaning that you don't have

00:03:57.740 --> 00:03:59.660 align:middle line:90%
clean-cut different events.

00:03:59.660 --> 00:04:03.530 align:middle line:84%
But they tend to go into a
continuous stream of sound,

00:04:03.530 --> 00:04:05.280 align:middle line:90%
exactly like in language.

00:04:05.280 --> 00:04:07.690 align:middle line:84%
And that, by the way, is
one of the reasons why

00:04:07.690 --> 00:04:09.590 align:middle line:84%
it's difficult to learn
foreign languages,

00:04:09.590 --> 00:04:12.515 align:middle line:90%
because speech is continuous.

00:04:12.515 --> 00:04:16.450 align:middle line:84%
In other words, speech is
coarticulated, as they say.

00:04:16.450 --> 00:04:20.010 align:middle line:84%
So you have to be able to
pick up the discrete events

00:04:20.010 --> 00:04:22.110 align:middle line:90%
from a continuous stream.

00:04:22.110 --> 00:04:25.490 align:middle line:84%
And in your research, I know you
have been working theoretically

00:04:25.490 --> 00:04:25.990 align:middle line:90%
on this.

00:04:25.990 --> 00:04:28.020 align:middle line:84%
But also, I know
that you're working

00:04:28.020 --> 00:04:30.480 align:middle line:84%
in the lab with experiments
on these things.

00:04:30.480 --> 00:04:32.260 align:middle line:84%
Can you just tell a
little bit about how

00:04:32.260 --> 00:04:34.760 align:middle line:84%
you're actually doing this and
what you're doing in the lab?

00:04:34.760 --> 00:04:38.785 align:middle line:84%
Yes, we try to figure out
exactly what musicians

00:04:38.785 --> 00:04:39.720 align:middle line:90%
are doing.

00:04:39.720 --> 00:04:43.290 align:middle line:84%
So far, we have, at
least for my research,

00:04:43.290 --> 00:04:46.960 align:middle line:84%
I've focused mostly on what
we call sound-producing body

00:04:46.960 --> 00:04:48.830 align:middle line:90%
motions in music.

00:04:48.830 --> 00:04:53.980 align:middle line:84%
So what we do is that we use
this so-called motion capture

00:04:53.980 --> 00:04:57.440 align:middle line:84%
technology, which is
essentially an infrared camera

00:04:57.440 --> 00:04:59.670 align:middle line:90%
system with markers.

00:04:59.670 --> 00:05:03.540 align:middle line:84%
And then we place markers
on the fingers, hands, arms,

00:05:03.540 --> 00:05:06.970 align:middle line:84%
shoulders, the whole torso,
head to feet, and so on,

00:05:06.970 --> 00:05:10.030 align:middle line:84%
depending upon what we are
interested in studying.

00:05:10.030 --> 00:05:13.180 align:middle line:84%
So in the example of
a piano performance,

00:05:13.180 --> 00:05:14.900 align:middle line:84%
it looks like
almost like a person

00:05:14.900 --> 00:05:18.440 align:middle line:84%
having smallpox with all
these markers on their hands.

00:05:18.440 --> 00:05:21.440 align:middle line:84%
And then we have very
detailed information

00:05:21.440 --> 00:05:23.840 align:middle line:84%
about how this
preparatory of motion

00:05:23.840 --> 00:05:26.710 align:middle line:90%
is going on all the time.

00:05:26.710 --> 00:05:30.250 align:middle line:84%
The next project will
be drumming, a drum set.

00:05:30.250 --> 00:05:32.350 align:middle line:84%
Because, as you know,
you have the tom-toms,

00:05:32.350 --> 00:05:33.810 align:middle line:84%
and you have the
right, and hi-hat,

00:05:33.810 --> 00:05:36.232 align:middle line:84%
the bass drum kicks,
snare, whatever.

00:05:36.232 --> 00:05:40.530 align:middle line:84%
And whatever rhythmical
pattern the drummer is playing

00:05:40.530 --> 00:05:43.540 align:middle line:84%
needs to have this
constant motion.

00:05:43.540 --> 00:05:47.290 align:middle line:84%
Because you hit the ride, and
then you are hitting the snare.

00:05:47.290 --> 00:05:51.220 align:middle line:84%
And the moment you hit the
ride, the stick passes off,

00:05:51.220 --> 00:05:54.960 align:middle line:84%
and you try to aim as best
you can for the next event.

00:05:54.960 --> 00:05:57.880 align:middle line:84%
So in that sense, you
always have this context.

00:05:57.880 --> 00:06:02.490 align:middle line:84%
And it also is mobilising the
rest of the body, the torso.

00:06:02.490 --> 00:06:05.670 align:middle line:84%
The drummer would sit
down on a stool like this

00:06:05.670 --> 00:06:11.460 align:middle line:84%
and try as best as he or she
can to exploit the rebound.

00:06:11.460 --> 00:06:13.360 align:middle line:84%
A drummer has to
conserve energy.

00:06:13.360 --> 00:06:15.840 align:middle line:84%
Otherwise, he or she would
be completely exhausted

00:06:15.840 --> 00:06:17.740 align:middle line:90%
after a couple of minutes.

00:06:17.740 --> 00:06:22.430 align:middle line:84%
So again, returning
to your main question,

00:06:22.430 --> 00:06:25.180 align:middle line:84%
you have all this
embodiment, how

00:06:25.180 --> 00:06:29.000 align:middle line:84%
the body is an integral part of
the music-making and the body

00:06:29.000 --> 00:06:30.300 align:middle line:90%
motion, of course.

00:06:30.300 --> 00:06:34.825 align:middle line:84%
And then somehow, the body has
to adapt to the constraints,

00:06:34.825 --> 00:06:38.934 align:middle line:84%
as we say, of the physics
of the musical instruments.

00:06:38.934 --> 00:06:40.570 align:middle line:90%
Well, thank you very much.

00:06:40.570 --> 00:06:42.130 align:middle line:90%
You're welcome.

00:06:42.130 --> 00:06:45.056 align:middle line:90%