This paper proposes a robot hand for a violin-playing robot and introduces a newly developed robot finger. The proposed robot hand acts as the left hand of the violin- playing robot system. The violin fingering plays an important role in determining the tone or sound when the violin is being played. Among the diverse types of violin fingering playing, it is not possible to produce vibrato with simple position control. Therefore, we newly designed a three-axis load cell for force control, which is mounted at the end of the robot finger. Noise is calculated through an analysis of the resistance difference across the strain gauge attached to the proposed three-axis load cell. In order to ensure the stability of the three-axis load cell by analyzing the stress distribution, the strain generated in the load cell is also verified through a finite element analysis. A sound rating quality system previously developed by the authors is used to compare and analyze the sound quality of the fourth-octave C-note played by a human violinist and the proposed robot finger.
Keywords: Anthropomorphic robot hand, Strain gauge, Finite element analysis, Three-axis load cell, Violin-playing robot, Violin fingering, Sound quality rating system.
Manuscript received Mar. 14, 2016; revised May 24, 2016; accepted June 17, 2016.
This research was supported by Technology Innovation Program of the Knowledge economy (No. 10041834, 10045351, 10067414) funded by the Ministry of Knowledge Economy (MKE, Korea), the Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education (No. 2012R1A1A2043822, 2014S1A5B6035098, 2016R1D1A1A02936946) and the Ministry of Science, ICT and Future Planning, Korea, under the Global IT Talent support program (H09051510020001002) supervised by the Institute for Information and Communication Technology Promotion.Hyeonjun Park (koreaphj@khu.ac.kr) and Donghan Kim (corresponding author, donghani@khu.ac.kr) are with the Department of Electrical Engineering, Kyung Hee University, Yongin, Rep. of Korea.
Bumjoo Lee (bjlee@mju.ac.kr) is with the Department of Electrical Engineering, Myongji University, Yongin, Rep. of Korea.
I. Introduction
Since the beginning of the 21st century, the rapid development and diversification of the functions of robots in the robotics industry have led to the classification of robots based on their varying roles. The International Federation of Robotics classifies a robots into the following three categories [1]:
1) Industrial robots
2) Service robots for professional use 3) Service robots for personal and private use
A number of studies have been conducted on human-robot interaction (HRI) with the aim of developing service robots for personal and private use. In particular, both the industry and researchers have focused on a variety of entertainment robots that can deliver information in a human-friendly way. Sensors such as vision, auditory, and touch sensors have been further installed in entertainment robots to identify the intended actions of human users. The rapid technological developments of vision and touch sensors in the field of entertainment robots have enabled the interaction of such robots with human users.
However, no suitable sensors or algorithms for recognizing human intent through auditory perception are yet available.
Currently, studies are actively being pursued for recognizing a variety of sounds and human voices [2], [3]. However, among the various studies for recognizing such sounds, studies specifically for investigating auditory feedback have been conducted to a limited extent in the medical field [4]. In particular, a self-treating method for people who stutter, which entails listening to one’s own voice and understanding its state, has been proposed [5]. Researchers have determined that this auditory feedback technology is the key element of HRI, and the present authors have accordingly developed a sound quality rating system for improving an existing violin-playing robot by
Development of Anthropomorphic Robot Finger for Violin Fingering
Hyeonjun Park, Bumjoo Lee, and Donghan Kim
combining this system with auditory feedback [6], [7]. The present work introduces a robot hand and a robot finger that perform the role of violin fingering for a violin-playing robot.
II. Background
1. Instrument-Playing Robot
Waseda University has developed the Waseda Saxophonist Robot [8] and Waseda Flutist Robot [9], which are played by controlling the flow of anthropogenic air using an air pump.
The significance of these robots is that they mimic a human lung However, these robots have a limitation in that they mostly simulate a mechanical mechanism similar to that of humans. As another type of instrument-playing robot, Toyota has developed a violin robot with precision control technology and provided a demonstration [10]. Ryukoku University has studied a violin robot to enable more natural playing by adding control commands that reflect the sensitivity of humans through the use of KANSEI data [11], [12]. However, because the played sound is not feedback into the control command of the robot, HRI can be achieved in a simple manner by means of a specified command. These robots are worthy examples demonstrating the excellence of robot hardware design and control technology, but they function by stringently executing pre-programmed control commands. The present authors have also developed a violin-playing robot by modeling a violin bow as a spring-damper model, and attempted to improve the robot’s sound quality by using the Q-score to evaluate the sounds being played [6], [7].
2. Manipulator End-Effector of Service Robot
Regardless of the field in which it is intended to be used, an end-effector is a necessary component for handling external objects in a manner similar to that by a human hand. In particular, accurate and precise control is important in the field of service robots that interact with humans. Thus, a considerable amount of research is being conducted on end- effector manipulation techniques [13]–[15]. The end-effector can be classified into two different types: gripper- and anthropomorphic-type end-effectors. The gripper of a home service robot developed by Lee and others [16] has three fingers and a total of eight degrees of freedom (DoFs).
Through an attachment of a reduced pressure sensor at the fingertip, flexible control is achieved when the gripper grasps an object. This gripper is highly effective for grasping circular objects, but it has a limitation in grasping and manipulating objects of various shapes, unlike the human hand. Because the gripper is made up of a link and gear structure, it may not survive when subjected to a strong external force and will
beome damaged during the gripping process. An open-source robotic gripper developed by Tlegenov and others [17] also has three fingers, where each finger has 2 DoFs. A single servo actuator is sufficient for driving the gripper owing to the combination of worm and spur gear. Because all movements can be controlled by a single servo motor, the gripper is inexpensive and light in weight. In addition, there is a low risk of damage even under the application of a strong external force because the gripper uses an elastic material such as a spring.
However, the gripper can conduct only simple tasks such as picking up an object, and it has limitations in cases where a machine needs to be operated or when tasks requiring precise operation need to be performed.
Because an anthropomorphic-type end-effector mimics the human hand and has the appearance of a multi-finger configuration, it can conduct a larger variety of tasks than a gripper-type end-effector. KITECH has developed KITECH Hand, which consists of four fingers including the thumb [18].
A DC motor with a built-in angle sensor is mounted in each of the finger joints. Force control is possible through the application of a feedforward current control method for an object, and compliance control is possible for external forces.
Because an actuator is mounted in each finger joint, this end- effector is expensive and contains a larger number of control elements. To overcome these problems, an underactuated robotic hand has been studied, where each joint is driven by wires and using a coupled-joint approach [19], [20]. Dalley and others [21] have developed a multi-grasp hand that has five fingers and a total of 16 DoFs. Motors are placed in the palm, and a wire-driven approach is used for driving the hand. A Hall sensor included in the motor is used to control the angular position of the robot hand. The robot hand is thus similar in appearance to the human hand and bones, and its size is similar to that of the hand of an adult male. This robot hand can grasp objects of various sizes and shapes, and is light in weight with a mass of 320 g. Mouri and others [22] similarly developed a robot hand that comprises five fingers and has a total of 16 DoFs. This hand is made up of joints coupled through an actuator and a four-bar planar mechanism. A six-axis force sensor is mounted onto each fingertip for accurate control, and a tactile sensor is mounted onto each finger and palm to secure the detection point.
The present paper proposes a robot hand that functions as the left hand of an existing violin-playing robot. The proposed robot hand will generate a more accurate sound by controlling the pressing force applied to a violin string during violin playing by the violin-playing robot. The effectiveness of the proposed robot hand was demonstrated by comparing the quality of sound generated by it with that generated through the violin fingering of a human violinist on the basis of an existing
sound quality rating system [7].
The rest of this paper is organized is follows. Section III briefly introduces the existing violin-playing robot system.
Section IV describes the proposed anthropomorphic-type robot hand and its finger used for violin fingering. Section V illustrates how violin fingering affects the actual violin sound quality and presents a comparison and analysis of the results of such sound quality for the case of violin fingering by a human violinist and the proposed robot finger when playing the fourth- octave C-note.
III. Previous Works
1. Concept of Violin Auditory Feedback
Auditory feedback is used as a stuttering therapy method in the medical field, and delayed auditory feedback (DAF) has been mainly studied.
DAF is a proven medical technique that reduces the stuttering symptoms by 70% by repeatedly recording the patient’s voice using a microphone and then allowing the patient to listen through headphones with 25 ms to 74 ms of delay, as shown in Fig. 1. If this auditory feedback is fused with visual and somatosensory feedback, it helps the patient talk or sing by allowing them to determine the accuracy of the sound.
Fig. 1. Principle of DAF.
This process has been utilized for the playing of music, and a previous work proposed a method for improving the sound quality automatically when using a violin robot. Figure 2 shows the framework of the auditory feedback system used in the violin robot. This system was implemented by listening and mimicking the playing technique of a violinist to improve the sound quality [7].
2. Characteristics of Violin-Playing Robot
We previously developed a violin-playing robot using an industrial six-axis vertical robot arm (RV-2SD), as shown in Fig. 3(a), to mimic the appearance of a human violinist. The overall height of the robot is 1,520 mm, and a robot arm is mounted on the shoulder. The robot arm is controlled using a PC through TCP/IP communication. The robot arm includes an AC servo motor, with a maximum load and maximum speed of 3 kg and 4,400 mm/s, respectively. An electric violin, which is robust to external noise, is used for an accurate analysis of the sound being played. Similar to the arrangement for a human violinist, the violin is fixed at an angle of 30° to the left shoulder. At the end of the robot arm a violin bow equipped with a force sensor is placed to measure the pressing and twisting forces. Figure 3(b) shows the overall control architecture of the developed violin-playing robot system. This architecture consists of a microcontroller unit (MCU) module, chart display, and PC application. The MCU module receives the performance information, including the bow force, bow speed, and sounding point, in real time. The chart display shows this performance information in real time. Finally, the PC application controls the robot arm [6], [7].
IV. Anthropomorphic Robot Finger for Violin Fingering When a human plays the violin, motion for plucking the
Fig. 2. Framework of auditory feedback system [7].
Pre-programed command
+ + – –
Control board
Force Speed Point Robot ARM
Frequency Magnitude
(Δ speed, Δ point, Δ force1, 2) (Δ force 1, 2)
Δ force Δ speed Δ point
Sound evaluation Auditory feedback
Violin playing
Force sensor 1, 2
Frequency Magnitude Force sensor 1 Force sensor 2 Robot control
Fig. 3. (a) Developed violin-playing robot system and (b) control architecture of the violin-playing robot [7].
Two-axis load cell
Position sensing Input
sound
Audio amp.
12-bit ADC
DMA ch 1
DMA ch 1
1024-sample FFT
Load cell amp.
Two-axis load cell ADC
IR sensor amp.
IR sensor ADC
Protocol
Timer 1 (4,000 Hz) Timer 2
Chart display Auditory feedback Peak freq. (Hz)
Peak mag. (dB) Load cell 1 (mV) Load cell 2 (mV) Sounding point
(n) Sound evaluation Q-factor
Harmonic Gaussian
Bow velocity
Bow force
Sound point Change position CORTEX-M3
Windows App.
TCP/IP
RV-2SD RS-232
(a) (b)
Violinist robot
Fig. 4. Mechanism of proposed robot hand design: (a) front view, yaw motion, (b) right-side view, and (c) joint for yaw motion.
197 mm 112 mm 100 mm 88 mm
Z
Y
Yaw
Pitch
(a) (b) (c)
X
bowstrings as well as a violin fingering technique for pressing the strings using fingers are required. This work focuses on the vibrato technique, which is a violin fingering technique implemented by performing light quivering action of the finger while pressing the strings. Realization of this motion requires the addition of a robot hand to the left hand of the violin- playing robot. As shown in Fig. 4(a), the designed robot hand is of the anthropomorphic-type, where the size of the hand is similar to that of a normal adult male. In order to implement all DoFs, a total of four motors are installed inside each finger and
a total of eight motors are mounted on the palm. In this way, each finger has 3 DoFs and the robot hand has a total of 12 DoFs, as indicated in Table 1.
1. Design of Robot Finger
The human finger consists of three joints: the metacarpophalangeal (MCP) joint between the palm and the first node, the proximal interphalangeal (PIP) joint between the first and second nodes, and the distal interphalangeal (DIP)
Table 1. Specifications of proposed robot hand.
Item Specifications
Total DoF 12 (except thumb)
Total weight 500 g
Length of robot hand 197 mm
Width of robot hand 86 mm
Number of joints 16
Actuator DC motor with reduction gear
Mechanism Wire driven
Fig. 5. (a) Structure of human finger, (b) designed robot finger with DoFs of robot finger, and (c) the manufactured robot finger.
X Z
Y
1 DoF for Z-axiz (dependent joint)
2 DoF for Y- and Z-axes
(a) (b) (c)
Fig. 6. Bending motion of robot finger.
DIP wire PIP wire DIP pulley
PIP pulley
DC Motor
Violin string
Violin finger board
Step 1 Step 2 Step 3
joint between the second and third nodes. The MCP joint has 2 DoFs—pitch and yaw—and the PIP and DIP joints have a dependent relationship, and are characterized by 1 DoF, each in the pitch direction. These characteristics are reflected in the proposed robot finger, as shown in Fig. 5(b). When the motor installed in the finger spins, both the DIP and PIP joints bend simultaneously, similar to the movement of a human finger.
Owing to the addition of a joint to permit yaw motion, it is possible to generate both pitch and yaw motions of the MCP joint. The motor installed inside the finger is a DC-geared motor, having a gear ratio of 324:1, maximum torque of 30 mNm, and maximum speed of 15 rpm. Figure 6 shows the
Fig. 7. Wire-driven tension adjustment structure.
Y Z X
Restraints of DIP wire Restraints of PIP wire
Bolt
Fig. 8. Measurement experiment for robot fingertip force.
Weighing indicator
Robot finger
Connection wire Load cell
process of the proposed robot finger pressing the violin string.
First, the internal structure of the robot finger is developed with one DC-geared motor as mentioned above. The DIP and PIP joints are both designed using a bearing and pulley structure.
When the internal DC motor is operated, the wire around the pulley becomes pulled and both the DIP and PIP joints bend simultaneously, as shown in step 2. Finally, the robot finger presses the violin string, as shown in step 3. When an external force is continuously applied during the application of the wire- driven method, a wire can stretch or loosen at the fixed part.
To solve this problem, the mechanism shown in Fig. 7 is employed in the proposed robot finger, allowing the wire to be tightened when it loosens. The DIP and PIP wires are fixed to their respective restraints, as shown in Fig. 7; these components are fastened and fixed by bolts.
When the bolt is rotated in either the clockwise or counter- clockwise direction, horizontal movement along the X-axis is possible. Because this mechanism permits an adjustment of the tension of the wires as required, it provides the advantage of eliminating the disassembly of the components to adjust the wire tension. In addition, we developed a system to measure the fingertip force of the proposed robot finger, as shown in Fig. 8. A load cell (maximum of 5 kg) is connected to the robot finger with a connection wire. Between 12 V to 24 V is applied when measuring the fingertip force, as shown in Table 2.
In addition, we developed a measurement system to measure the fingertip force of the proposed robot finger, as shown in Fig. 8. A load cell (maximum of 5 kg) is connected to the robot
Table 2. Specifications of proposed robot hand.
Input voltage (V) Fingertip force (N)
12 1.1
13 1.4
14 1.6
15 1.8
16 2.0
17 2.1
18 2.2
19 2.3
20 2.4
21 2.5
22 2.6
23 2.7
24 2.8
finger with a connection wire. Between 12 to 24 V is applied to measure the fingertip force, as shown in Table 2.
2. Three-Axis Load Cell for Finger Tip
In regard to the force control of the robot finger, an analysis of the load direction of the robot fingertip is needed. When a human violinist executes a vibrato, it is accomplished in a step- by-step manner, as shown in Fig. 8. First, as shown in Fig. 9(a), the finger is secured to the fingerboard as the violin strings are being pressed. Then, the finger is straightened, and by slightly sliding the finger back and forth, the strings are lightly shaken, as shown in Fig. 9(b). Throughout this process, the strings must continue to be pressed with the same amount of force. In other words, in order to implement the vibrato playing, the directional strength of the pressed string as well as the amount of vertical force being applied must be measured. Additionally, the violin-playing robot system when used by the robot hand has a yaw motion. Therefore, when operating the yaw motion, measurement of the applied directional force must be possible.
To maintain the precise amount of force required for pressing the violin string, this force has to be measured and the finger has to be controlled. Thus, the end joint of the finger was designed using a three-axis load cell, and a strain gauge is bonded to its surface, as shown in Fig. 10. Table 3 summarizes the characteristics of the designed three-axis load cell and strain gauge [24]. When a load is applied to the load cell, the binocular structure of the load cell deforms and the output reading of the strain gauge changes.
In addition, we developed a measurement system to measure
Fig. 9. Step-by-step procedure of vibrato playing.
(b) (a)
Fig. 10. Proposed three-axis load cell.
Z X
Y
Strain gauge (b) (a)
Table 3. Specifications of three-axis load cell and strain gauge.
Item Scale
Load cell specifications
Material Aluminum 6061-T6
Poisson ratio 0.33
Yield strength 2.7 × 108 N/m2 Strain gauge specification
Gauge factor 2.35 2%
Gauge resistance 120.4 0.5 Ω
Transverse sensitivity 3.5%
the fingertip force of the proposed robot finger, as shown in Fig. 8. A load cell (maximum of 5 kg) is connected to the robot finger with a connection wire. Between 12 V to 24 V is applied to measure the fingertip force, as shown in Table 2. Because the magnitude of the electrical signal is small, two strain gauges are configured as a half-bridge circuit, and its output is amplified, as shown in Fig. 11. The relationship between the supply voltage and the voltage measured from the half-bridge circuit is represented as follows:
1 1 21
2 11 22 33 44Δ
Δ Δ Δ
R ,
R R R R R
e E
R R R R
R R
(1)
where R1 and R2 are the attached strain gauges, and R3 and R4
are fixed resistors. The output from the circuit, e, is amplified using the amplifier, converted using an A/D converter, and transmitted to the PC through RS-232 communication. The proposed three-axis load cell considers the following two conditions:
1) An error in the measured strain in a direction other than the direction of the load application to the load cell.
2) Safety factor when a load is applied to the load cell.
When the strain gauge is attached to the target object for measurement, the resistance value of the strain gauge shows changes in a direction other than the direction of the applied load. These components act as noise in the measured strain signal. Thus, the noise is analyzed on the basis of the resistance difference across the strain gauge attached to the proposed three-axis load cell. The resistance change of the strain input is represented as follows:
a a t t s s,
R K K K
R
(2)
where εa is the normal strain along the axial direction, εt is the transverse direction of the strain gauge, and γs is the shearing strain. Further, Ka, Kt, and Ks are the gauge factors of the strain gauge for the axial, transverse, and shearing strains, respectively. Because the shearing strain is generally considered to be a very small value, it is ignored, and the above condition is represented as follows:
a a t t s s
t
a a t s
a
t
a a t a
a
, where 0,
1
,
, wh ree .
R K K K
R K K
K
K vK v
K
(3)
In (3), the values of Kt/Ka and v represent the traverse sensitivity and Poisson’s ratio, respectively, as indicated in Table 3. Calculation of these values indicates that the calculated strain is 1.16% smaller than the actual measured strain. The complex system is strongly influence by an error of 1%, but proposed system is not significantly affected.
When the proposed robot finger presses the violin string, the load is measured in the three-axis load cell. If the deformation is excessive, it affects the overall structure of the three-axis load cell. Thus, the strain generated in the object is verified using FEA, and subsequently, the stability of the three-axis load cell is confirmed through an analysis of the stress distribution. This study evaluated the design stability using the factor of safety (FoS), S, which is based on the maximum von Mises stress.
The FoS for assessing the design stability of the three-axis load cell is expressed as follows:
Y VM
Y
2 2 2 2 2 2 2
,
3( ) ,
x y z x y y z z x xy yz zx
S
(4) where VM is the maximum von Mises stress, and is its Y yield stress. Table 3 summarizes the physical properties of the material used for the analysis. Because a violinist generally applies a force of 2.5 N to press a string for fingering, this value is set as the external load. Figure 12 shows the results of a static analysis based on this condition. The analysis results indicate that the minimum FoS values in the X-, Y-, and Z-directions are 13.71, 16.47, and 16.94, respectively, as shown in Table 4.
Most of the mechanical part designs employ an FoS value of 2 to 3 in the case of a static analysis, and this value increases to 10 in the case of a dynamic analysis [25]. Thus, it was confirmed through the results of the FEA that the proposed three-axis load cell has sufficient stability to press a violin string.
Fig. 11. Half-bridge circuit and amplification system.
Amp.
R2
R4
R1
R3
E
e
Output sig.
Fig. 12. FEA of three-axis load cell (applied force, 2.5 N): (a) X-, (b) Y-, and (c) Z-directions.
Y Z X
Y Z X
Z Y X
(a) (b) (c)
Table 4. FEA results.
Item X-direction Y-direction Z-direction Max. von Mises
Stress (MPa) 16.70 20.10 16.20
Factor of safety 13.71 16.47 16.94
Fig. 13. (a) Flowchart of output check of load cell, and (b) test environment of three-axis load cell.
3-axis load cell
Load cell Amp.
12 bit ADC
Text display
Graph display
STM32F407 Window App.
RS-232
(a)
(b)
Robot finger built in load cell
Magnitude of load cell
Fig. 14. Three-axis load cell test when built into fingertip: (a) hard and (b) elastic objects [26].
(a)
(b)
To determine whether there is normal output when the manufactured load cell operates the robot finger, the system shown in Fig. 13(a) was constructed. At every scheduled time, the MCU transmits the amplification value of the three-axis load cell to the dot net based Windows application through RS- 232 communication.
In this application, the real-time value of the load cell is
expressed through a text and graphic interface. At the moment the actual violin string is pressed, the stiffness is strong, and in the process of continuously pressing the string, an elastic force is applied to the robot finger through the tension of the string.
Therefore, the process of the robot finger pressing a strong object and an object with elastic force progresses. Along with Fig. 13(b), the results of developing and constructing the testing environment can be seen in Fig. 14. When the robot finger presses each of the two objects, the magnitude can be verified through the Windows application. This signifies that the designed load cell system is operating normally.
V. Experimental Results of Violin Fingering
A violinist creates a scale by pressing four fingers of the left hand, excluding the thumb, on the string corresponding to the pitch; this configuration of the left hand is known as a position, as shown in Fig. 15(a). The position is an important aspect because the accuracy, tone, and volume of the sound depend on the angle and pressing force on the strings, which is known as fingering. Violin fingering enables the creation of four notes on each string by shortening the string length at a single position.
The lowest position is called the first position, and the position usually goes up to the seventh position. As shown in Fig. 15(b), the playing scale changes depending on the finger location.
Before the proposed robot finger can apply violin fingering, a suitable pressing force to be applied to the strings needs to be
Fig. 15. (a) Violin fingering position and (b) violin fingering map.
E A D G
F# G A B
B C D E
E F# G A
A B C D
1st finger 2nd finger 3rd finger 4th finger (b)
(a)
Fig. 16. Three classifications of fingering force [27].
Case 2 Case 1
Case 3 Fingerboard
Fingerboard
Fingerboard Firmware device of sound
quality rating system
Table 5. Frequencies of the musical scale (Hz).
Scale Octave
C D E G
1 34.7 36.7 41.2 49.0
2 65.4 73.4 82.41 98.0
3 130.8 146.8 164.8 196.0
4 261.6 293.7 329.6 392.0
Fig. 17. Results of sound quality: plot of (a) fpeak and (b) Mpeak.
10 20 30 40 50 0
200 400 600 800 1,000
Sample index
10 20 30 40 50 0
20 40 60 80
Sample index Case 1 Case 2 Case 3 Mpeak
fpeak
(a) (b)
Table 6. Average values of fpeak and range of Mpeak (human violinist).
Case Average value of fpeak (Hz) Average range of Mpeak (dB)
1 261.7 48.6–85.2
2 773.9 39.6–74.0
3 807.2 36.6–70.0
determined. Thus, this study classifies the physical pressing force applied to a string as derived from a human violinist into three categories. The played sound is then analyzed using the sound quality rating system developed previously by the authors.
Figure 16 shows the three conditions of fingering to determine the pressing force required to be applied to the violin strings to generate an accurate note. Case 1 is where the violin string is pressed to the fingerboard, case 2 is when the string is pressed so as to not touch the fingerboard, and finally, case 3 is when the finger is only laid on the strings. In this experiment, a violinist also played to increase the reliability of the test. At this point, the reference note is the fourth-octave C-note. Figure 17 shows the evaluation results obtained using the sound quality rating system. Table 6 summarizes the average values of fpeak
and the range of Mpeak for all three cases, where fpeak denotes the frequency of sound and Mpeak denotes its magnitude.
Table 5 shows the frequencies of the musical scale, where
Fig. 18. Comparison results of sound quality in the case of violin playing by a human violinist and by the proposed robot finger: plot for (a) robot finger and (b) robot finger with polyurethane foam [28], [29].
20 40 60 80
250 255 260 265 270
Sample index
20 40 60 80
40 50 60 70 80 90
Sample index Robot finger Human
20 40 60 80
250 255 260 265 270
Sample index 40 20 40 60 80
50 60 70 80 90
Sample index Robot finger Human
fpeak Mpeak
fpeak Mpeak
(a) (b)
(c) (d)
Table 7. Average values of fpeak and range of Mpeak (robot finger with poly-urethane foam and human violinist).
Case Average value of fpeak (Hz) Average range of Mpeak (dB)
Robot finger 262.6 51.4–83.2
Human
violinist 261.7 48.6–85.2
the frequency of the played fourth-octave C-note is 261.6 Hz.
Thus, it can be seen through the experiment results that the force of case 1 is appropriate for an accurate sound generation.
This value is 2.5 N, which is equivalent to the pressing force generated by a human violinist. Because the fingertip force of the proposed robot finger is 2.8 N at 24 V, as shown in Table 2, the violin strings can be pressed with a sufficient amount of force. The results of the fourth-octave C-note generated by a human violinist and by the proposed robot finger were compared. A violin is played through the production of vibration from the friction of the bowstrings. Because the proposed robot finger is made of metal, the vibration caused by the friction of the string and the bow with the robot finger causes resonance, which results in the generation of an inaccurate sound, as shown in Fig. 18(a). In the case of the human violinist, the bone of the hand is surrounded by the outer skin, which reduces the vibration caused by the friction of the string and bow, thereby generating an accurate sound. Thus,
polyurethane foam was attached to the surface of the robot finger and the experiment was repeated, the results of which are shown in Fig. 18(b). The results show in particular that the value of fpeak is consistent with the corresponding value obtained from the human violinist. However, the amplitude of sound of the robot finger is still comparatively weaker than that of the human violinist, as indicated in Table 7. Therefore, in this regard, additional support is needed.
VI. Conclusion
This paper introduced the implementation of a proposed left robot hand of an existing violin-playing robot system. The proposed robot hand was designed to enable violin fingering to accurately generate various sounds. From among the various types of violin fingering techniques available, the proposed robot hand focuses on the vibrato technique. In order to implement violin fingering actions, the robot hand was designed to be of an anthropomorphic-type, and its size was similar to that of the hand of a normal adult male. The vibrato technique enables the addition of a joint for achieving a yaw motion of the robot hand. One motor was installed in each finger, and eight motors were mounted on the palm, resulting in a total of 12 DoFs.
A human finger consists of MCP, PIP, and DIP joints, where the PIP and DIP joints have a dependent relationship and 1 DoF in the pitch direction. These characteristics were reproduced in the proposed robot finger through a pulley and wire-driven mechanism design. However, because the wire- driven mechanism has the disadvantage of lengthening the wire owing to the application of a constant external force, the proposed robot finger was designed with a mechanism that permits an adjustment of the wire tension.
To achieve the violin fingering actions, the fingertip force must be measured in real time to maintain an appropriate amount of force for pressing the violin strings. Thus, the last joint of the finger was designed as a three-axis load cell and strain gauges were bonded to its surface. The strain error of the proposed three-axis load cell was 1.16%, but the system was not significantly affected. The load cell has sufficient design stability to press the violin strings, which was verified through an FEA.
The results of generating the fourth-octave C-note by a human violinist and by the proposed robot finger were compared using a sound quality rating system developed previously by the present authors. Because the bone of the human hand is surrounded by skin, the vibrations from the friction between the bow and string are reduced, which eliminates any resonance and results in an accurate playing of a particular note. After the attachment of polyurethane foam to
the surface of the robot finger, the results obtained for the robot finger became consistent with those of a human violinist.
Future work will include the design of a mechanical system that will permit the application of a vibrato technique to a violin-playing robot and implementing force control in the robot finger through a calibration of the three-axis load cell proposed in this study
References
[1] International Federation of Robotics, Accessed Sept. 7, 2015.
http://www.ifr.org/service-robots/
[2] T. Shibata, T. Tashima, and K. Tanie, “Emergence of Emotional Behavior through Physical Interaction between Human and Robot,” IEEE Int. Conf. Robot. Autom., Detroit, MI, USA, May 10–15, 1999, pp. 2868–2873.
[3] I.B. Yildiz, K. Von Kriegstein, and S.J. Kiebel, “From Birdsong to Human Speech Recognition: Bayesian Inference on a Hierarchy of Nonlinear Dynamical Systems,” PLoS Comput. Biol., vol. 9, no. 9, Sept. 2013, pp. 1–16.
[4] P.G. Blanchet and P.R. Hoffman, “Factors Influencing the Effects of Delayed Auditory Feedback on Dysarthric Speech Associated with Parkinsons, Disease,” J. Commun. Disorders, Deaf Stud.
Hearing Aid, vol. 2, no. 1, Mar. 2014, pp. 1–11.
[5] S. Cai, Online Control of Articulation Based on Auditory Feedback in Normal Speech and Stuttering: Behavioral and Modeling Studies, Ph.D. dissertation, Massachusetts Institute of Technology, Feb. 2011, pp. 185–209.
[6] B.C. Min et al., “Improvement of Violinist Robot Using a Passive Damper Device,” J. Intell. Robot. Syst., vol. 72, no. 3, Jan. 2013, pp. 343–355.
[7] W.S. Jo et al., “A Study on Improving Sound Quality of Violin Playing Robot,” Int. Conf. Autom., Robot., Applcat., Queenstown, Australia, Feb. 17–18, 2015, pp. 185–191.
[8] J. Solis, A. Takanishi, and K. Hashimoto, “Development of an Anthropomorphic Saxophone-Playing Robot,” Brain, Body, Mach., vol. 83, no. 1, Mar. 2010, pp. 175–186.
[9] J. Solis et al., “The Development of the Anthropomorphic Flutist Robot at Waseda University,” Int. J. Humanoid Robot., vol. 3, no.
2, June 2006, pp. 127–151.
[10] Y. Kusuda, “Toyota’s Violin-Playing Robot,” Ind. Robot: Int. J., vol. 5, no. 6, Dec. 2008, pp. 504–506.
[11] K. Shibuya, S. Matsuda, and A. Takahara, “Toward Developing a Violin Playing Robot-Bowing by Anthropomorphic Robot Arm and Sound Analysis,” IEEE Int. Symp. Robot Human Interactive Commun., Jeju, Rep. of Korea, Aug. 26–29, 2007, pp. 763–768.
[12] K. Shibuya, T. Asada, and S. Sugano, “An Algorithm to Convert KANSEI Data into Human Motion,” IEEE Int. Conf. Syst., Man, Cybern., San Diego, USA, Oct. 14, 1998, pp. 1190–1194.
[13] M. Salerno et al., “A Novel 4-DOFs Origami Enabled, SMA
Actuated, Robotic End-Effector for Minimally Invasive Surgery,”
IEEE Int. Conf. Robot. Autom., Hong Kong, China, May 31–June 7, 2014, pp. 2844–2849.
[14] M.G. Catalano et al., “Adaptive Synergies for a Humanoid Robot Hand,” IEEE Int. Conf. Humanoid Robot., Osaka, Japan, Nov. 29, 2012, pp. 7–14.
[15] J. Butterfass et al., “DLR-Hand II: Next Generation of a Dextrous Robot Hand,” IEEE Int. Conf. Robot. Autom., Seoul, Rep. of Korea, May 21–26, 2001, pp. 109–114.
[16] Y.C. Lee et al., “Development of the Robot Gripper for a Home Service Robot,” ICCAS-SICE, 2009, Fukuoka, Japan, Aug. 18–
21, 2009, pp. 1551–1556.
[17] Y. Tlegenov, K. Telegenov, and A. Shintemirov, “An Open- Source 3D Printed Underactuated Robotic Gripper,” IEEE/ASME Int. Conf. Mechatronic Embedded Syst. Applic., Senigallia, Italy, Sept. 10–11, 2014, pp. 1–6.
[18] J.H. Bea et al., “Development of a Low Cost Anthropomorphic Robot Hand with High Capability,” IEEE Int. Conf. Intell. Robots Syst., Vilamoura, Algarve, Portugal, Oct. 7–12, 2012, pp. 4776–
4782.
[19] B. Massa et al., “Design and Development of an Underactuated Prosthetic Hand,” IEEE Int. Conf. Robot. Autom., Washington, DC, USA, May 11–15, 2002, pp. 3374–3379.
[20] Y. Kurita et al., “Human-Sized Anthropomorphic Robot Hand with Detachable Mechanism at the Wrist,” Mechanism Mach.
Theory, vol. 46, no. 1, Jan. 2011, pp. 53–66.
[21] S.A. Dalley et al., “A Multigrasp Hand Prosthesis for Transradial Amputees,” Auun. Int. Conf. IEEE. Eng. Medicine Biol. Soc., Buenos Aires, Argentina, Aug. 31–Sept. 4, 2010, pp. 5062–5065.
[22] T. Mouri, H. Kawasaki, and S. Ito, “Unknown Object Grasping Strategy Imitating Human Grasping Reflex for Anthropomorphic Robot Hand,” J. Adv. Mech. Design, Syst., Manuf., vol. 1, no. 1, Jan. 2007, pp. 1–11.
[23] H.J. Park et al., “A Study about Sound Quality for Violin Playing Robot,” Procedia Comput. Sci., vol. 56, pp. 496–501.
[24] H.J. Park et al., “Development of Robotic Finger Using 3-Axis Load Cell for Violin Playing Robot,” Adv. Sci. Technol. Lett., vol.
90, Apr. 2015, pp. 22–26.
[25] Wikipedia, the Free Encyclopedia, Accessed Sept. 7, 2015.
https://en.wikipedia.org/wiki/Factor_of_safety
[26] H. Park, “Robot Finger for Violin Fingering” Filmed, Feb. 2016, YouTube Video, 0:20, Posted, Feb. 2016. https://youtu.be/
fiX4FyelSUM
[27] H. Park, “Violin-Playing: Case 1-3 Fingering Force,” Filmed, May 2016, YouTube Video, 0:26, Posted, May 2016.
https://youtu.be/ Q192rKptlUg
[28] H. Park, “Robot Finger Violin-Playing, Fingering Force Case 1,”
Filmed, May 2016, YouTube Video, 0:17, Posted, May 2016.
https://youtu.be/JADNfEJ7e3g
[29] H. Park, “Robot Finger (attached poly-urethane foam) Violin-
Playing, Fingering Force Case 1,” Filmed, May 2016, YouTube Video, 0:14, Posted, May 2016. https://youtu.be/9pLKXtNNlsE
Hyeonjun Park received his BS degree in robotics engineering from Hoseo University, Seoul, Rep. of Korea in 2014, and his MS degree in electronics and radio wave engineering, Kyung Hee University, Yongin, Rep. of Korea in 2016. His research interests include human-robot interaction, musical instrument playing robots, and music (sound) recognition.
Bumjoo Lee received his BS degree in electrical engineering from Yonsei University, Seoul, Rep. of Korea in 2002, and his MS degree and PhD in electrical engineering from Korea Advanced Institute of Science and Technology, Daejeon, Rep. of Korea in 2004 and 2008, respectively. Since 2012, he has been with the Department of Electrical Engineering, Myongji University, Yongin, Rep. of Korea, where he is currently an assistant professor. His research interests include humanoid robotics, especially regarding motion planning and control algorithms.
Donghan Kim received his BS and MS degrees and his PhD in electrical engineering from Korea Advanced Institute of Science and Technology in 1995, 1998, and 2003, respectively. He has been with the Department of Electronics and Radio Engineering at Kyung Hee University, where he is currently an associate professor. He was a postdoctoral fellow at the University of Illinois, Urbaba-Champaign in 2003. He was the recipient of the first award of the humanoid robot competition at FIRA robot soccer in 2002.
His current research interests include human-robot interaction, particularly violin-playing robots.
