• 검색 결과가 없습니다.

RNN (Recurrent Neural Network)

N/A
N/A
Protected

Academic year: 2021

Share "RNN (Recurrent Neural Network)"

Copied!
18
0
0

로드 중.... (전체 텍스트 보기)

전체 글

(1)

Recurrent

Neural

Network

Wanho Choi

(2)

Handling Sequence Data

For example, speech recognition

FCNN, CNN, AE, and GAN?

RNN!

The current state ➜ the next state

y

1

x

1

y

2

x

2

y

n

x

n

h

1

h

2

h

n−1

output

hidden

input

h

t

= f( h

t−1

, x

t

)

(3)

x

t

fold

unfold

: sequence length

n

y

0

x

0

x

1

Vanilla RNN Structure

y

1

y

t

h

1

h

2

h

n−1

y

n−1

x

n−1

h

t

(4)

x

t

fold

unfold

: sequence length

n

y

0

x

0

x

1

Vanilla RNN Weights

W

hy

W

hy

W

hh

W

hh

W

hh

W

xh

W

xh

y

1

y

t

h

1

h

2

h

n−1

y

n−1

x

n−1

W

hy

W

xh

h

t

W

hh

W

hy

W

xh

(5)

x

t

h

t

= tanh( W

hh

h

t−1

+ W

xh

x

t

)

y

t

= W

hy

h

t

W

hh

W

hy

W

xh

y

t

tanh +

Vanilla RNN Computation

(6)

How to use RNN in PyTorch

rnn = torch.nn.RNN( input_size, hidden_size )

outputs, status = rnn( input_data )

<creation>

<run>

x

t

y

t

input shape = ( batch_size, sequence_length, input_size )

output shape = ( batch_size, sequence_length, output_size )

(= hidden)

(7)

Example: ‘hello’

import torch

import numpy as np

torch.manual_seed(0) # to make results deterministic and reproducible

unique_characters = ['h', 'e', 'l', ‘o'] INPUT_SIZE  = len(unique_characters) # = 4 HIDDEN_SIZE = len(unique_characters) # = 4 x_data = [[0, 1, 2, 2]] # hell # batch_size = 1 x_one_hot = [[[1, 0, 0, 0],  # ‘h'       [0, 1, 0, 0],  # ‘e'       [0, 0, 1, 0],  # ‘l'       [0, 0, 1, 0]]] # ‘l' y_data = [[1, 2, 2, 3]] # ello x = torch.FloatTensor(x_one_hot) y = torch.LongTensor(y_data)

1/2

(8)

Example: ‘hello’

# batch_first guarantees the order = (batch_size, sequence_length, data_size)

model = torch.nn.RNN(INPUT_SIZE, HIDDEN_SIZE, batch_first=True)

CostFunc = torch.nn.CrossEntropyLoss()

optimizer = torch.optim.Adam(model.parameters(), lr=0.1) for i in range(10):

    outputs, _ = model(x)

    cost = CostFunc(outputs.view(-1, INPUT_SIZE), y.view(-1))     cost.backward()

    optimizer.step()

    optimizer.zero_grad()

    result = outputs.data.numpy().argmax(axis=2)

    result = ''.join([unique_characters[c] for c in np.squeeze(result)])     print(result) # ello

(9)

Example: ‘hello’

one hot encoding

‘h’ = [1,0,0,0]

‘e’ = [0,1,0,0]

‘l’ = [0,0,1,0]

‘o’ = [0,0,0,1]

(10)

RNN is flexible.

one-to-one

one-to-many

many-to-one

many-to-many

vanilla RNN

image captioning

video captioning

image → sentence

(seq. of words)

language modeling

sentence →sentiment

translation

(11)

Stacked RNN

time

depth

(12)

RNN Applications

Temporal analysis

: time-series anomaly detection and time-series prediction

Computer vision

: image description, video tagging and video analysis

NLP

: sentiment analysis, speech recognition, language modeling, machine translation and text

generation

https://missinglink.ai/guides/neural-network-concepts/cnn-vs-rnn-neural-network-right/

CNN vs RNN: Which Neural Network Is Right for You?

RNN CNN Hybrids

(13)

Object Recognition using RCNN

(14)
(15)
(16)
(17)

Advanced RNN

LSTM (Long Short Term Memory)

GRU

https://www.youtube.com/watch?v=-SHPG_KMUkQ

(18)

참조

관련 문서

시간에 종속되는 통행 속도의 특 성을 반영하기 위해 적용되었던 시계열 분석 기법인 Autoregressive Integrated Moving Average(ARIMA)와 순환 신경망(Recurrent

Degree Centrality The counts of how many connections a node has Popularity or influence of a node (e.g., word, person) in the network Betweenness Centrality The extent to which

Four volumes of the text are used as a core text to instruct third and fourth year elementary school students in the rudiments of Korean language related to daily

Motivation – Learning noisy labeled data with deep neural network.

Human Actions Tracking and Recogni tion Based on Body Parts Detection via Artificial Neural Network.. ○ 인공 신경망 -&gt;

Human Actions Tracking and Recogni tion Based on Body Parts Detection via Artificial Neural Network.. ○ 인공 신경망 -&gt;

A Study on the Evaluation of Space Recognition Space Recognition Space Recognition in the Space Recognition in the in the in the Concourse of Hospital using the

In this chapter, we introduce unsupervised pattern recognition using a spiking neural network (SNN) based on the proposed synapse device and neuron circuit.. Unsupervised