• Recent updates • Questions? Projects? Today-Oct. 31

Neural Prosthetic Engineering

Today- Oct. 31 ^st

• Questions? Projects?

• Review

– Speech Processing

• Recent updates

– By major commercial CI manufacturers – New Technologies

1

Neural Prosthetic Engineering

2

Review

Neural Prosthetic Engineering

Human Speech Generation

3

• Vocal fold generates sound that consists of fundamental and harmonic frequencies (source of sound)

• Vocal tract modifies amplitudes of these frequency

components to make the sound distinguishable from

others (articulation by vocal tract).

Neural Prosthetic Engineering

Consonants/vowels/formants

4

Consonant and vowels

• Articulations at various places in vocal tract

• Roughly speaking,

• Vowels are lower frequency sounds

• Consonants are higher frequency sounds

• Formants are distinct frequency bands in the sound spectrogram

• F1 and F2 may be used to represent vowels

• Consonants may need additional higher frequency formants to be distinguished

• F0 is the fundamental frequency of the pitch

Neural Prosthetic Engineering

How we hear

5

• Temporal (Rate) code theory

• Place code theory

• Volley theory

• We use both Temporal(Rate) and Place Cues in hearing.

Neural Prosthetic Engineering

Hearing by Cochlear Implant

6

• Extracted features are used

• F0/F1/F2

• Analog waveforms are used

• Compressed Analog

• CIS uses extracted features and biphasic pulse

stimulation at a constant rate

Neural Prosthetic Engineering

What to improve

7

1.

2.

Neural Prosthetic Engineering

Development of Present and future

Cochlear Implant Products and Performance

8

Neural Prosthetic Engineering

CI Milestones

• 1960 Single channel CI on three patients by William House

• 1976 A Multichannel CI on two patients by Graeme Clark

• 1977 Bilger report confirmed effectiveness of multichannel CI

• 1985 Nucleus 22 became the first Multichannel CI approved by FDA

• 1991 CIS speech processing strategy (Blake Wilson)

• 1997 Clarion (Advanced Bionics) device approved by FDA

• 2001 Medel device approved by FDA

• 2000’s EAS and Bilateral CI Other milestone

• 1997 DBS (Activa manufactured by Medtronic) approved by FDA as a treatment for essential tremor and Parkinson's disease.

Dystonia in 2003, and OCD in 2009.

• 2013 Argus II retinal implant (manufactured by Second Sight Medical Products) approved by FDA

9

F.G.Zeng et al., (IEEE Review in Biomedical Engineering, 2008)

Neural Prosthetic Engineering

History of the Cochlear Implant

• Pioneers

– Andre Djourno and Charles Eyries (in Paris, 1957)

• Eyries implants Djourno's induction coils in two patients

• Alternating current transmitted to the coil produces perception of sound

• Early Developments in the Western Hemisphere

– William House, John Doyle, James Doyle (Los Angeles, 1960)

• Effect electrical stimulation during stapes surgery

• Implant 3 patients with a single gold electrode – Blair Simmons (Stanford University, 1964)

• Develops a six-electrode system using a percutaneous plug ( Ineraid) – Robert Michelson (San Francisco, 1970)

• Implant 3 patients using a gold two-electrode system ( Advanced Bionics) – William House (Los Angeles, 1972)

• First wearable cochlear implant device using a centering coil and magnet

• House/3M single channel cochlear implant (approved by the FDA in 1984)

Djourno (Physiologist)

Eyries (ENT surgeon)

William House (ENT Surgeon)

House/3M single channel device

10

Neural Prosthetic Engineering

History of the Cochlear Implant

• Development of a Mutichannel Device (1970-80s)

– Single channel device  Very Poor speech understanding – Competition

• Michelson, Merzenich, Robert Schindler (UCSF)  Advanced Bionics Corp.

• Hochmair (Vienna, Austria)  Med-El GmbH.

• Graeme Clark (The University of Melbourne in Australia)

– Research supported by public donation (commenced 1967)

– First Multichannel Cochlear Implant Patient (1978)  Cochlear Ltd.

• FDA Approved Multichannel CI Manufacturers

– Cochlear (Australia) – 1985

– Advanced Bionics (Austria) – 1996

– Med-El (Austria) – 2001 (1994 – European release)

• Lasker~DeBakey Clinical Medical Research Award (2013)

– Graeme M. Clark, Ingeborg Hochmair and Blake S. Wilson

• For the development of the modern cochlear implant

- a device that bestows hearing to individuals with profound deafness.

Rod Saunders (First multi- channel CI patient) and

Graeme Clark

11

Neural Prosthetic Engineering

Product History

12

1978 (by Michelson)

A.A.Eshraghi (The Anatomical Record, 2012)

1973 (by House, single channel)

Neural Prosthetic Engineering

Product History

13

1985 (Cochlear Ltd. Nucleus, FDA approval, multi-channel)

www.cochlear.com

1989 (Cochlear Ltd.

Mini speech processor)

Neural Prosthetic Engineering

Product History

14

1997 (Cochlear Ltd. Sprint, Digital signal processor (DSP))

www.cochlear.com

2002 (Cochlear Ltd. Nucleus®

24 Contour Advance™, design for structure preservation)

Neural Prosthetic Engineering

Product History

15

www.cochlear.com

2008 (Cochlear Ltd. Hybrid,

hearing aid + cochlear implant) 2009 (Cochlear Ltd. Nucleus 5)

2015 (Cochlear Ltd. Nucleus 6, smart speech processor)

Neural Prosthetic Engineering

Two Recent Advances

16

• Bilateral electrical stimulation

• Combined electric and acoustic stimulation (EAS) for patients with residual, low-frequency hearing

http://www.otosurgery.org/

Neural Prosthetic Engineering 17

http://www.otosurgery.org/

Results with bilateral

implants using independent processors, Müller et al., 2002

Sentences, Noise Right

right both left

Percent Correct

0 20 40 60 80 100

Sentences, Noise Left

right both left

Monosyllabic Words, No Noise

right both left

32 86 96 107 118 148 151 257 298 Subjects

right both left

Percent Correct

0 20 40 60 80 100

Implant(s)

right both left right both left

Neural Prosthetic Engineering 18

http://www.otosurgery.org/

Combined Electric Acoustic Stimulation(EAS)

■ Combined EAS: Hearing Aid (HA) + Cochlear Implant (CI) on same ear

■ Many implant candidates  Good low-frequency hearing but poor high frequency hearing

■ Low-frequency  Acoustic Hearing using a HA High-frequency  Electrical Hearing using a CI

■ Good speech perception in noisy environments

■ Latest EAS technique

- Surgery: the round window approach (conventional method: cochleostomy) - Electrode: flex, Short & thin electrode (half insertion)

Hybrid CI Device (HA+CI) Ski-slope type

SNHL

Sentence discrimination tests for a EAS patient

In quiet SNR 15 dB SNR 10 dB

W. Gstoettner et al. (Acta Oto-Laryngologica, 2004) Hearing Aid

(HA)

Neural Prosthetic Engineering

Performance of Cochlear Implant

19

B. S. Wilson and M. F. Dorman (IEEE Trans Biomed Eng, 2007)

Test

Star patient’s score

B. S. Wilson & M. Dorman (Hearing Research, 2008)

Percent correct scores for 55 CI users

Neural Prosthetic Engineering 20

Cochlear Implant Manufacturers

■ Big 3 companies

- Cochlear, Australia - Med-El, Austria

- Advanced Bionics, USA

■ Others

- Oticon(Former Neurelec), France - Nurotron, China

- Todoc, S. Korea

Neural Prosthetic Engineering 21

Cochlear Ltd.

■ Nucleus series - Market leader

■ Dual microphones - Directionality

■ Wireless control

- Interaction with TV and smartphone

■ Colorful designs - Users satisfaction

■ Water-resistance - Living convenience

■ Electric acoustic stimulation (EAS) - High-performance

■ Data logging - Rehabilitation

www.cochlear.com

Neural Prosthetic Engineering 22

Cochlear Ltd.

■ Thinnest cochlear implant among 3 major manufacturers

■ Advanced off stylet electrode array - Perimodiolar electrode array

- Near zero insertion force

www.cochlear.com

Neural Prosthetic Engineering 23

Cochlear Ltd.

■ Smart speech processor (SmartSoundiQ)

- Integrated hybrid mode - Wireless connectivity

- Data logging and analysis: information for next fitting - Speech in noise, wind, quiet, and music modes

- Dual microphones

- Wind noise reduction (WNR)

www.cochlear.com

Neural Prosthetic Engineering 24

Med-El

■ Synchrony series

- Freely rotating and self- aligning magnet

■ EAS

■ Water-resistance

■ Color options

■ Test in rainy environment

■ Wind noise reduction

■ 3.0 T MRI (with magnet) - Other groups 1.5 T MRI

www.medel.com

Neural Prosthetic Engineering 25

Med-El

www.medel.com

Neural Prosthetic Engineering 26

Med-El

www.medel.com

■ Various types of electrode array - Long electrode: for stimulation full cochlea

- Short electrode: for EAS

■ Structure preservation - Atraumatic insertion - Reimplantation

Neural Prosthetic Engineering 27

Advanced Bionics

www.advancedbionics.com

■ Water-resistance

■ Wireless control

■ Colorful design

■ Dual microphones

■ Acquired by Sonova in 2009/ Co-working with

Phonak (hearing aid company)

■ EAS – FDA approval in Aug.

2015

Neural Prosthetic Engineering 28

Advanced Bionics

www.advancedbionics.com

■ HiFocus mid-scala electrode array

- Pre-curved electrode

Neural Prosthetic Engineering

New Technologies

29

Neural Prosthetic Engineering

Next Version of Cochlear Implant?

30

www.cochlear.com

www.cochlear.com

Candidates

1. Silicon-based CI

2. Polymer- based CI 3. Optical

stimulation

…

Neural Prosthetic Engineering

Silicon-Based Device

31

• Advantages of silicon-based neural implant – Batch process

• High-yield

• Mass production – MEMS technology

• Miniaturization

• High-density

• Highly functional device

• Integration

• Disadvantages of silicon-based neural implants – Brittleness

– Stiffness

– Long-term reliability

Neural Prosthetic Engineering

Silicon-Based Device

J.Wang and K.D.Wise, J.MEMS 2009. 32

J.Wang and K.D.Wise, J.MEMS 2008.

Neural Prosthetic Engineering

Polymer-Based Neural Implant

33

• Conventional neural implants – Titanium package

• Highly hermetic

• MR image artifact problem

– Wire-based electrode array

• Manual fabrication, limited integration density of contacts

• Polymer-based neural implants – Thin & compact

– Simpler manufacturing process

• MEMS technology – No MR image artifact

– Relatively low hermeticity than metals

10 mm

S. K. An et al., IEEE-TBME 2007.

Neural Prosthetic Engineering

Liquid Crystal Polymer (LCP)

34

• Two widely used polymers – Polyimide, parylene-C

• Relatively high water absorption rate

• Lack of long-term reliable

• Liquid Crystal Polymer:

A New Biomaterial for Implantable Devices

– Biocompatible

– Chemically inert and mechanically stable – Flexible

– Compatible with MEMS technologies – RF transparent

– Very Low water absorption rate (<0.04%) – Fusion-bondable

– Deformable

LCP^[1]

(Vecstar)

Polyimide ^[2]

(PI2525)

Parylene-C^[3]

(GALXYL)

Melting Temp. (°C) 280~335 ^{>400 290}

Tensile Strength

(MPa) 270~500 128 69

Young’s Modulus

(GPa) 2~10 2.4 3.2

Water absorption

(%) ^{< 0.04 2.8 0.06 ~ 0.6}

Dielectric Constant

(@1MHz) 2.9 3.3 2.95

[1] Kuraray group, http://www.kuraray.co.jp/en/

[2] HDMicroSystems, http://hdmicrosystems.com/HDMicroSystems/en_US/

[3] V&P Scientific, Inc., http://vp-scientific.com/parylene_properties.htm

Neural Prosthetic Engineering

LCP-Based Cochlear Implant

35

Neural Prosthetic Engineering

Electronics Design

36

• Simplified circuit block diagram

Contact pad side Coil side

Fabricated electronic board for cochlear implant

[1] S. K. An et al., IEEE TBME 2007.

Current stimulator chip

 0.8 μm high voltage CMOS process (AMS)

 CIS strategy

 16 ch. mono- & 15 ch. bi-polar stimulation

 Stimulation rate : 1,000 pps/channel

 Duration : 0 μs ~ 56 μs (7 levels, 8 μs step)

 Amplitude : 0 μA ~ 1.8 mA

(255 levels, 7.3 μA step )

[2] J. Jeong et al., Conf Proc IEEE Eng Med Biol Soc 2011.

• Fabricated on Copper clad LCP film – Planar type receiving coil integration

for low-cost & miniaturized cochlear implant

Neural Prosthetic Engineering

• LCP-based Cochlear Implant System

Small & Light Package

37

[1] S. K. An et al., IEEE-TBME 2007.

Parameter Value

Carrier Frequency 2.5 MHz Operating Distance 12 mm

Current Amplitude 8 μA~1.8 mA Pulse Duration* 0~56 μs

Pulse Rate* Total 8,000 Hz

Parameter *LCP-based CI Titanium-based CI^[1]

Package Size 20 x 28 mm² 65.7 x 33.3 mm² Package

Max. thickness (w/o magnet)

1.2 mm 8.2 mm

Weight

(w/o magnet) 0.45 g 10.4 g

1. Physical Dimension (LCP vs. Ti-based CI)

2. Wireless & Stimulation

* LCP-based cochlear

implant Metal-based

cochlear implant^[1]

Neural Prosthetic Engineering

Flexible LCP-Based Cochlear Electrode Array

38

• LCP-based cochlear electrode array

– Multi-layered & tapered LCP film structure – Sawtooth-like structure

– Silicone elastomer encapsulation

• Minimal insertion force

• Atraumatic insertion

Cross Section of

16-Channel Interconnection

`

`

`

` 300

350255075

750

575

LCP Substrate First LCP Cover

Second LCP Cover

Multi-layered Lead Wires Via Unit: μm

[1] K. S. Min et al., Otology & Neurotology 2014.

[2] T.M. Gwon et al., Biomedical Microdevices 2015.

Sites E-Gun Deposited/

Electroplated

Blind Via

Side Via Opening

Lead Wires

38

Neural Prosthetic Engineering

Evaluation – Human Temporal Bone Insertion Study

39

• Safety validation of the electrode

– Human temporal bone insertion test – Insertion depth measurement from CT

scan image of the temporal bone

• No trauma at basal turn

• Dislocation into scala vestibule at middle turn in 630°insertion trial

• Max. insertion depth: ~630°

Cochlear electrode

[1] T.M. Gwon et al., Biomedical Microdevices 2015.

CT scan images (▲ )and cross-section (▼) of the temporal bones

(round window approach:

insertion depth ~ 630°)

CT scan images (▲ )and cross-section (▼) of the temporal bones

(cochleostomy approach:

insertion depth ~ 500°)

SV: Scala vestibuli ST: Scala tympani

SV

SV SV

ST ST

ST

ST

ST ST

SV

SV SV

Neural Prosthetic Engineering

MRI Compatibility and In Vivo Functionality Testing

40

•

– LCP-based CI: Little MR image artifact

50 mm

Axial Coronal

* *

3.0 T MR images of the head

[1] J. H. Kim et al., Clin Exp Otorhinolaryngol 2010.

• LCP-based Cochlear Implant

– EABR measurement

0 2 4 6 8

-6 -4 -2 0 2 4 6

Voltage (V)

Time (ms) Measured eABR

Wave V

Stimulus artifact Package

Electrodes

Stimulation:

800 μA, 32 μs (EABR: electrically evoked auditory

brainstem response)

Neural Prosthetic Engineering

Long-Term Reliability of LCP

41

• Accelerate Soak Test in 75℃ Phosphate Buffered Saline (PBS) Solution

• Multi-Interdigitated Electrode (IDE) array

• Long-term leakage current measurements of polyimide, parylene-C, and LCP.

[1] SW Lee et al., IEEE-TBME 2011

0 15 30 45 60 75 90 105 120 135 150

10^-13 10^-12 10^-11 10^-10 10^-9 10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2

Soak Time (days)

Current (A)

Polyimide

Ch1 Ch2 Ch3 Ch4 Ch5 Ch6

0 15 30 45 60 75 90 105 120 135 150

10^-13 10^-12 10^-11 10^-10 10^-9 10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2

Soak Time (days)

Current (A)

Parylene-C

Ch1 Ch2 Ch3 Ch4 Ch5 Ch6

0 30 60 90 120 150 180 210 240 270 300 330 360 390 420 450 480 10^-13

10^-12 10^-11 10^-10 10^-9 10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2

Soak Time (days)

Current (A)

LCP

Ch1 Ch2 Ch3 Ch4 Ch5 Ch6

Mean time to failure

> 1 year (380 days)

0 15 30 45 60 75 90 105 120 135 150

10^-13 10^-12 10^-11 10^-10 10^-9 10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2

Soak Time (days)

Current (A)

Polyimide

Ch1 Ch2 Ch3 Ch4 Ch5 Ch6

0 15 30 45 60 75 90 105 120 135 150

10^-13 10^-12 10^-11 10^-10 10^-9 10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2

Soak Time (days)

Current (A)

Parylene-C

Ch1 Ch2 Ch3 Ch4 Ch5 Ch6

0 30 60 90 120 150 180 210 240 270 300 330 360 390 420 450 480 10^-13

10^-12 10^-11 10^-10 10^-9 10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2

Soak Time (days)

Current (A)

LCP

Ch1 Ch2 Ch3 Ch4 Ch5 Ch6

0 15 30 45 60 75 90 105 120 135 150

10^-13 10^-12 10^-11 10^-10 10^-9 10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2

Soak Time (days)

Current (A)

Polyimide

Ch1 Ch2 Ch3 Ch4 Ch5 Ch6

0 15 30 45 60 75 90 105 120 135 150

10^-13 10^-12 10^-11 10^-10 10^-9 10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2

Soak Time (days)

Current (A)

Parylene-C

Ch1 Ch2 Ch3 Ch4 Ch5 Ch6

0 30 60 90 120 150 180 210 240 270 300 330 360 390 420 450 480 10^-13

10^-12 10^-11 10^-10 10^-9 10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2

Soak Time (days)

Current (A)

LCP

Ch1 Ch2 Ch3 Ch4 Ch5 Ch6

Mean time to failure : 75 days

Mean time to failure : 115 days

Polyimide Parylene-C LCP

Neural Prosthetic Engineering

Optical Stimulation

42

• Introducing Optics in cochlear implantation

• Multichannel Optical Cochlear Implantation – Highly local stimulation

– Fine frequency resolution

– Increasing the number of effective channels

– Limited by line-of-sight property of light, and added complexity in optical instrumentation

Optical stimulation of auditory neurons: effects of acute and chronic deafening.

Richter CP, Bayon R, Izzo AD, Otting M, Suh E, Goyal S, Hotaling J, Walsh JT Hearing research 2008 Aug; 242(1-2):42-51

http://openoptogeneticsblog.org/?p=682

Neural Prosthetic Engineering

Reference

• J. Jeong et al., Conf Proc IEEE Eng Med Biol Soc 2011.

• S. K. An et al., IEEE-TBME 2007.

• T.M. Gwon et al., Biomedical Microdevices 2015.

• K. S. Min et al., Otology & Neurotology 2014.

• A.A.Eshraghi (The Anatomical Record, 2012)

• B. S. Wilson & M. Dorman (Hearing Research, 2008)

• Ducci et al., Otology & Neurotology, 2010

• J.Wang and K.D.Wise, J.MEMS 2008.

• C.-P. Richter et al., Hearing Research, 2008

• L.E.Moreno et al., Hearing Research 2011.

• M. Jeschcke et al., Hearing Research 2015.

43