Reconfigurable Free-Space

(1)

O ^PTICAL S ^TORAGE A ^REA N ^ETWORKS

Tim D. Wilkinson, Bill Crossland, Neil Collings, Fan Zhang, and Mark Fan, Cambridge University

Reconfigurable Free-Space

Optical Cores for Storage Area Networks

I

NTRODUCTION

Storage area networks (SANs) are currently one of the growing applications of modern packet- based networks. The need to store vast amounts of data is ever increasing as the importance of the Internet as an information knowledge base increases. Already storage requirements in excess of petabytes are required by database systems and search engines across the global Inter- net, and this requirement is expanding as the Internet becomes the information resource of choice within the home [1].

The role of the SAN is to provide access to mass storage devices as efficiently as possible, h e n c e r a t h e r t h a n r e m a i n i n g a p e r i p h e r a l device they have become an integrated part of the network structure [2, 3] and protocol [4].

As with any network, its efficiency in transmit- ting data is a function of not only data rate but also the ability to switch data between different ports or nodes within the network [5].

The problem of switching becomes even more critical in SANs as the requirement to access stored data with minimal delay (or latency) is what defines its performance. Traditional optical switching methodology has been based on minimizing packet loss or data loss during network transmission. This is still a vital requirement in SANs, but now we must also take into account the latency of the switching process in order to access stored data as efficiently as possible.

One of the main problems in the definition of switching in SANs is that is depends heavily on the size and structure of the SAN itself.

Definitions of SANs range from the intercon-

nection of individual storage devices within a box or cabinet [3] to the interconnection of farms of routers in an IP network such as might be accessed by a search engine [5]. What is common to all these potential SAN structures is that the interconnection technology is rapid- ly heading toward optical fiber links to allow data rates in excess 10 GHz to be utilized either at the storage level or to interconnect storage banks. One common SAN implementation is to use the Fibre Channel (FC) system to interconnect cabinets or racks of storage devices at a data rate of 2.5 GHz using multimode optical fiber.

Given these transmission rates and the use of optical fiber technology, the problem or bottleneck becomes the chosen method of switching used to move packets between cabinets or nodes in the SAN. The use of electronic switching has proven successful in systems to date; however, this is already pushing silicon very large-scale integration (VLSI) and printed circuit board (PCB) design to its limit, and the step to 10 GHz data rates will only push the technology and cost further. A more desirable approach is to switch the packets in the free-space or fiber optical domain [6, 7], as has been demonstrated in the past for both asynchronous transfer mode (ATM) [8–11]

and IP [12, 13] based optical switches. The choice of optical switch core depends heavily on the application and implementation of the SAN as this will define the performance crite- ria such as latency and packet loss. Figure 1 shows three possible implementations of an optical switch core within three potential SAN architectures.

The possible architectures shown in Fig. 1 can all be implemented using either fiber- based or free-space optical switch cores. The fiber-based wavelength-division multiplexed (WDM) system is described in the SAN paper [14], so here we concentrate on free-space optical switch core technologies for SAN applications. These can be separated into two cate- gories: holographic beam steering switches [15] and optical shutter-based switches [8–13].

Within this article we highlight the two free- space switching mechanisms with particular reference to their application in SAN structures.

A

BSTRACT

The role of the optical storage area network is becoming increasingly important in computer network architectures, with the amount of information being presented and requested on the Internet ever increasing. One of the key ques- tions is what is the role of, and how do we control, an optical switch core within a SAN to optimize its performance. In this article we review two potential optical switch technologies and assess their performance as a SAN optical switch core.

(2)

H

OLOGRAPHIC

B

EAM

S

^TEERING

S

^WITCHES

Optically transparent switching has become a vital part of the modern vision of telecommunications transmission networks. This has included the use of both micro-electrome- chanical (MEMs) beam steering switches[8]

as well as liquid crystal (LC) holographic switches [15]. Integral to these fiber-to-fiber a p p l i c a t i o n s i s a n o p t i c a l l y t r a n s p a r e n t switch that is polarization-insensitive and capable of low loss and low crosstalk. In a packet-based SAN system, the role of these s o r t s o f s w i t c h e s i s y e t t o b e f u l l y d e t e r - mined; however, given the popularity of FC i n t h e s e S A N s , t h e r e m u s t b e a s e r i o u s

advantage in using beam steering switches, as shown in Fig. 2.

In an LC-based beam steering switch, the active element is a reconfigurable grating displayed as a phase only object. This is used to direct the input light to the desired port, as shown in Fig. 2. Each input port can be either an optical fiber or laser source, and the output port can also be a fiber or photodetector. If sin- gle-mode fiber is used (as in transmission networks), the scalability is limited by the launch condition at the output port fiber; however, using multimode fibers, multimode fiber rib- bons, or photodetectors at the output, this constraint is greatly reduced, leading to scalable switch structures.

The results demonstrated in Fig. 3 show how the light from a fiber can easily be controlled and directed in a very efficient manner, but in order to achieve this level of performance the switching speeds have been low (on the order of milliseconds). The requirements of a SAN are mostly over short distances, so the loss of the switch is less of a constraint, which opens up the possibility of using faster, less efficient LC electro-optic effects such as fast ferro- electrics or electroclinics. This combined with the use of multimode sources and detectors such as multimode optical fibre or vertical cavity surface emitting laser (VCSEL) arrays and photodetectors makes the use of holographic beam steering a definite technology for optical switch cores in SANs.

As a demonstration of the integration of a free-space holographic beam steering system, a prototype 1 × 8 optical switch [15] (shown in Fig.

4) was built using a reflective silicon backplane containing a custom designed LC material designed for low-loss operation. The SLM was designed as a 540 × 1-pixel silicon backplane with 20 µm pitch pixels and 2 µm pixel deadspace.

The switch consists of four main components: a waveguide array acting as the interface between the fibers and the switch; a Fourier lens; a reflective binary-phase LC device that acts as a beam- steering element; and a custom interface board.

n n n

nFigure 1.Implementations of an optical core in a SAN structure: a) a board-based rack system; b) a Fibre Channel system; c) a WAN or MAN router farm.

Optical switch

Optical

switch MAN/WAN

Router Router

Fibre Channel

(a) (b) (c)

Storage units or line cards

Router Router

nnn

nFigure 2.The basic structure of a holographic beam steering switch.

Output fiber array

Input fiber array

Phase hologram array

(3)

S

^HUTTER

-B

^ASED

O

PTICAL

S

WITCHING

A shutter-based optical switch uses a blocking modulator to allow access to output ports via an optical interconnect. A typical shutter-based switch is shown in Fig. 5. The input array can be either optical fibers or lasers such as VCSELs, which are arranged in a k×karray (total number of ports is N= k²).

The input array is then optically fanned out using a fixed hologram to/from a k×karray of the input array. This gives a total of k²×k²(or N× N) input channels in the shutter plane, each of which corresponds to a particular shutter. Within each replicated array of inputs, all but one channel are blocked, and the position of the replicated array of the inputs corresponds to a particular output port. The final process is to fan-in each replication to each particular output port.

The key technology in this type of architecture is the choice of shutter as this defines the switching speed in the system. There are many possible options including LCs, MEMS, and Pockels effect modulators such as lead lan- thanum zirconium titanate (PLZT) [16]. It is often perceived that a technology is not fast enough; however, packets are rarely switched as individual entities in classical packet networks such as IP. This is an important consideration in SANs as the protocol is much simpler in terms of how the switching is implemented as the nodes are fixed and of a known number. Even with this in mind, it is important to note that switching speeds can be slower than the packet rate as groups of packets are switched over long periods relative to the packet time itself. Hence, for many SAN implementations, switching speeds on the order of 1 µs with modest buffering can be of importance as long as the interconnect is optically transparent and highly parallel.

We have considered two possible modulation technologies for the shutters, LC and PLZT.

Both LC and PLZT modulators are not quick enough to be used to directly modulate the light at high-speed data rates, but they can be placed in large parallel arrays to increase the aggregate bandwidth, as demonstrated by the opto-RAM LC over silicon chip, which used 4096 LCl mod-

ulators switching in 10 µs to create a high aggregate throughput [6]. The opto-RAM formed the input plane to a free-space optical switch for ATM. The disadvantage of slow speed is bal- anced by the ability to display massive amounts of information in parallel in an optical format.

Figure 6 shows a microscope picture of the 12- transistor SRAM memory/pixel arrangement and test images.

The trade-off between switching speed and parallelism is the limiting factor in using LC modulators as the input medium in optical interconnects. Suitable LC materials can easily be found to switch down to 1 µs, and there are materials that have been measured with switch- nnn

nFigure 3.Diffracted orders of a binary phase hologram (38 percent efficien- cy).

Camera coordinate (mm)

1.00 1.25

0.00 0 Recorded camera intensity (arb.) 25

50 75 100 125 150 175 200 250 275

0.75 0.50

0.25

nnn

nFigure 4.A 1 × 8 switch demonstrator based on FLC SLM. (The LC device can be seen on the leftmost side).

nnn

nFigure 5.Fan-out and fan-in processes in a shutter-based optical switch.

Input

Fan out

Reconfigurable shutter

Fan in

Output

Shuttered replication

(4)

ing times down to 100 ns [17]. The main problem with this approach was the level of parallelism (4096²channels after fanout) required to image packets in a large optical switch, as each packet had to be displayed in parallel on the input chip. The best technology for doing this is VCSELs which can be made in arrays with com- plementary metal oxide semiconductor (CMOS) drivers and are capable of gigahertz modulation rates.

A comparison of switching speed between a ferroelectric LC shutter and a PLZT shutter is shown in Fig. 7. Both are capable of getting to the 3 µs target, by raising either the temperature or switching voltage. There are also new poly- meric nonlinear optical (NLO) materials being produced [18] that offer modulation rates in excess of 100 GHz and could be integrated into silicon-based modulation structures. As the desired data rate increases, the justification for optical interconnects becomes stronger and

stronger, and the desired interconnect distance becomes shorter. There is already a strong case for chip-to-chip interconnects in both telecommunications and computer systems, and even connections within the chip itself may have to be optical soon if suitable integratable technologies can be found.

There have been demonstrations of LC reconfigurable optical switches using a VCSEL array at the input [10]. The SAN optimized core switch system we are developing is based on a hybrid arrangement of optical and electronic switches as shown in Fig. 8, forming a direct path equiva- lent to a multistage Clos network [11]. The switch is designed to combine the parallelism of the optical interconnect with the speed and buffering of electronic switches to provide a powerful hybrid switch system with low overall switch latency. Neither the optics nor the electronics are pushed to their limits, creating both an economical solution as well as room for future nnn

nFigure 6.The Opto-RAM: a) SRAM pixel; b) pixel array.

a) b) c)

nnn

nFigure 7.Comparison between a) ferroelectric LC, DRA2; and b) PLZT shutter switching speeds.

Voltage (V)

11 12

6 2 4

Response time (µs)

6 8 10 12 14

10 9

8 7

Temperature = 30˚

Temperature = 40˚

Temperature = 50˚

Applied field (v)

130 140 60

0 0.5

Response time (µs)

1 1.5 2 2.5

120 110 100 90 80 70

Falling time Rising time

(5)

expansion. Based on a 256 × 256 port model, this switch is capable of data throughput approaching 1 Pb. If we assume the large data access requirements and port counts of a typical SAN, this may well be the sort of capacity required at peak times.

The operation of the switch in Fig. 8 can be implemented in many ways for any number of different packet switch configurations, but the key element is the optical switch core and how it is reconfigured. If it is assumed that there are 256 input ports each in sectors of 16 ports and running at 2 GHz data rate, a short packet such as an ATM packet or a control IP packet would be switched every 200 ns in the extreme case.

Internet traffic is not only made up of short packets, but also long ones (1500 bytes) and ATM cells can be aggregated in small numbers using buffers, so a switch with a slower reconfiguration rate can be used equally efficiently. The simulations shown in Fig. 9 with real bursty Internet backbone traffic data demonstrates that a switch reconfiguration rate of 1 µs is perfectly acceptable, with very little packet loss penalty [19]. In fact, this rate also works with low loss penalty when complex Internet traffic such as a community of interest’s traffic is applied.

The key parameter in setting up an optical switch for a SAN application in not so much packet loss as packet delay (or switch latency) in the switch. Packet latency is dominated by the time spent in the input buffer while waiting to be switched. It is measured from the moment a packet enters the input buffer to when the packet first reaches the core of the switch. Zero delay means the packet was servedas soon as it arrived at the buffer. Figure 10 shows how SLM reconfiguration time and m/n(speedup) affect the mean and maximum packet delay time and also

the percentage of packets with no delays. Note that the size of input buffers used in this simulation is quite large. This is because we are inter- ested in seeing how long packets are queued before they are served, and therefore we want to minimize buffer overflows (packet loss).

As expected, packet delays behave in the same way as packet losses. Increasing the m/n ratio increases the switch resources (number of paths in the interconnect) in the core and also nnn

nFigure 8.A hybrid electronic/optical packet switch using VCSELs and a liquid crystal shutter.

Output electronic module (OEM) (one per sector)

Output packets

Photodetector array (Bi/CMOS) VCSEL

Input electronic module (IEM) (one per sector)

VCSEL VCSEL VCSEL

VCSEL Array

Free-space optical switch (with in/out sectors)

Optical switch

Input packets

nnn

nFigure 9.Switch simulation of packet loss vs. reconfiguration rate. No ports

= 128, group size at output n= 8, m/nrepresents the speedup (extra paths through interconnect) required.

SLM switching speed (µs)

Total number of packets = 5,120,000 Switch dimension 128 x 128 Group size, n = 8

Input buffer size = 10 kbytes x 128 Segment size = 200 bytes Traffic load = 90%

0.6528

1.00E-09

Packet loss rate

1.00E-10 1.00E-08 1.00E-07 1.00E-06 1.00E-05 1.00E-04 1.00E-03 1.00E-02 1.00E-01 1.00E+00

0 1.30561.95842.6112 3.264 3.91684.56965.22245.87526.528

m/n = 1 (no buffer) m/n = 1.5 m/n = 2 m/n = 2.5 m/n = 3

m/n = 1 (with buffer) m/n = 1.5

m/n = 2 m/n = 2.5 m/n = 3

m/n = 1 m/n = 1.5 m/n = 2 m/n = 2.5 m/n = 3

(6)

the number of packets an output module can cope with at any given time. This improves the throughput of the switch and hence reduces delay. Faster SLM reconfiguration speed reduces the rest timeof the SLM interconnect and thus has the same effect on switch throughput.

A normal guide in maximum switch latency of this scale is in the region of 50 µs. Simulation results from Fig. 10 suggest that with m/n= 2.5 and SLM reconfiguration speed = 0.6528 ~ 3.264 µs, maximum packet delays fall well within this 50 µs mark. And over 80 percent of these packets experience no delay at all. For m/n= 2, the requirement for SLM reconfiguration time increase to 0.6528 ~ 2.6112 µs. For m/n= 1.5, required SLM reconfiguration time is at 0.6528 µs. The dotted square in Fig. 10 highlights the values of required m/nvalue and SLM reconfiguration rate in order to meet the 50 µs maximum delay mark.

C

ONCLUSIONS AND

D

ISCUSSION With the ever increasing demand on Internet resources and storage capacity, SANs will soon suffer from data bottlenecks and cost inflation due to limited switching capabilities within pure electronics. The efficiency of SAN systems has been increased by the use of optical technologies such as Fibre Channel to increase the available data bandwidth and transfer rate. The move to the optical domain has put pressure on the ability to switch optical data at high rates with low latency; thus, the bottleneck has shifted into the optical system. The use of a free-space optical switching core can ease this limitation through the exploitation of parallelism inherent in free- space optics.

There are two potential optical free-space switch core technologies that offer different performance parameters which depend heavily on the exact implementation and scale of the SAN.

Optical beam steering using liquid crystal holographic elements is optically simple and scalable, could be made fast for low latency switching, but

at the cost of overall link power inefficiency.

Even considering a reconfiguration speed on the order of milliseconds, there are possible applications within SAN systems. The access requirements for a Fibre-Channel-based SAN (as in Fig. 1b) show that the switching between banks of storage elements over the Fibre Channel can be at length scales in excess of 1 second, so a slow but efficient switch such as a holographic system can be used. If we consider the design of SANs distributed across a MAN or WAN (as in Fig. 1c) using multiply interconnected nodes, there is also scope for slower switching speed in the optical core. A slowly variable reconfigurable optical network architecture could be a key advantage in the process of partitioning storage elements across a MAN or WAN.

The second type of free-space optical switch is based on blocking or shutter switching. This offers faster switching and therefore lower latency, but is optically more complex to implement and more difficult to control. As has been proven by the simulation and experimental fabrication of the shutter based optical cores, a reconfiguration rate of 1msec is both achievable and acceptable. This allows an optical core switch within individual SAN boxes as shown in Fig. 1a, so it could be used with almost any existing optical SAN system such as those proposed on Fibre Channel, gigabit Ethernet, or multiprotocol label switching (MPLS). The exact choice of the most suitable switching technology is uncertain and application-dependent; however, what is certain is that both types of optical switch cores could solve the next big problem on the horizon of optical SANs.

R

EFERENCES

[1] P. Lyman and H. R. Varian, “How much information,”, University of California at Berkeley, 2003, http://www.sims.berkeley.edu/how-much-info/

[2] J. Tate et al., “Introduction to Storage Area Networks,” IBM Redbooks, SG24-5470-01, http://www.ibm.com/redbooks [3] Nortel Networks, “Storage Connectivity: A Guide to Stor-

age Networks and Their Optical Extension Across the Metropolitan Area,” http://www.nortelnetworks.com

nnn

nFigure 10.Simulation of latency vs. shutter speed for an optical packet switch.

SLM reconfiguration time (µs) Maximum delay (µs)

3.264 10

100 1000 10,000

0

2.6112 1.9584

1.3056 0.6528

SLM reconfiguration time (µs) Maximum delay (µs)

Total number of packets = 1,024,000 Switch dimension 128 x 128 Group size, n=8

Input buffer size = 1 Gbyte x 128 Segment size = 200 bytes Traffic load = 90%

3.264 10

100

100 1000 10,000

0

2.6112 1.9584

1.3056 0.6528

m/n = 1.5 m/n = 2 m/n = 2.5

(7)

[4] R. Barker and P. Massiglia, Storage Area Network Essentials: A Complete Guide to Understanding and Implementing SANs, Wiley, 2001.

[5] “Storage Networking 101,” Cisco white paper, http://

www.cisco.com

[6] R. J. Mears et al., “Telecommunications Applications of Ferroelectric Liquid Crystal Smart Pixels,” IEEE J. Sel.

Topics in Quantum Elect., vol. 2, no. 1, 1996, p. 35.

[7] S.-S. Lee et al., “Free-Space Fiber-Optic Switches Based on MEMS Vertical Torsin Reply to: On Mirrors,” J. Light- wave Tech., vol. 17, Jan. 1999, pp. 7–13.

[8] A.C. Walker et al., “Design and Construction of an Optoelectronic Crossbar Switch Containing a Terabit Per Second Free-Space Interconnect,” IEEE J. Sel. Topics in Quantum Elect., vol. 5, 1999, pp. 236–49.

[9] A. L. Lentine et al., “Asynchronous Transfer Mode Dis- tribution Network by Use of an Optoelectronic VLSI Switching Chip,” Appl. Opt., 36, 1997, pp. 1804–14.

[10] C. W. Wilmsen et al., “Vertical Cavity Surface Emitting Laser Based Optoelectronic Asynchronous Transfer Mode Switch,” Opt. Eng., vol. 38, no. 7, 1999, pp. 1216–22.

[11] R. W. A Scarr et al., “Highly Parallel Optics in ATM Switching Networks,” IEE Proc Optoelect., vol. 144, no.

2, 1997, pp. 53–60.

[12] Y. Wang, W.A. Crossland, and R. W. A. Scarr, “Mod- elling for Optically Interconnected Packet Switches,”

Proc. SPIE, 4213, 2000.

[13] C. A. T. H. Tee et al., “A Reconfigurable VCSEL Array Link Suitable for Use in Large Electronic IP Switches,”

Tech. Dig. OECC 2000, paper PD1-5, July 2000, Makuhari Messe, Tokyo, Japan, pp. 10–11.

[14] J. M. H. Elmirghani and I. H. White, “WDM Systems and Protocols for SAN Applications,” IEEE Commun. Mag.

[15] W. A. Crossland et al., “Holographic Optical Switching:

The ‘ROSES’ Demonstrator,” J.LT,vol. 18, no. 12, 2000, p. 184.

[16] F. Zhang et al., “Design of a Free-space Interconnec- tion Switch,” EOS Top. Mtg.: Optics in Comp., Engel- berg, Switzerland, 2004, pp. 55–56.

[17] H. Xu et al., “A Simple Method of Optically Enhancing the Small Electrooptic Response of Liquid Crystals,”

Appl. Phys. Lett., vol. 74, no. 21, pp. 3099–3101.

[18] L. R. Dalton et al., “J. Mater,” Chem., 1999, vol. 9, p. 1905.

[19] C. H. M Fan et al., “SLM Reconfiguration Time in Opti- cally Interconnected Packet Switch,” Proc SPIE.

B

IOGRAPHIES

TIMD. WILKINSON([email protected]) is a lecturer at Cambridge University Department of Engineering. He gained a B.Eng. (electrical) from Canterbury University, New Zealand, and was awarded a Ph.D. from Cambridge.

He is currently a fellow of Jesus College, Cambridge. He is involved in developing new applications for liquid crystal technology including interconnects, telecommunications, optical comparators, adaptive optics, and next-generation LCOS technologies. He is responsible for SLM and liquid crystal devices.

BILLCROSSLANDis a professor of photonics at Cambridge University Engineering Department. He has been involved with photonic switching for 15 years. He has been a pio- neer of ferroelectric liquid crystal devices for both displays and information processing, and has been co-originator of FELC/VLSI technology. In recent years a main interest has been in free space photonic switching systems. He has been author or co-author of in excess of 150 technical papers and 50 patents.

NEILCOLLINGSreceived a B.A. Honors degree in natural sci- ences from Cambridge University in 1971, and a Ph.D. degree in physics from Salford University in 1977. He has been responsible for programs on optical correlators, ferroelectric liquid crystal OASLMs, smart SLMs, optical neural networks, nonlinear semiconductor devices, and optical system design, with particular emphasis on optical interconnects and 3D display. He is a Fellow of the Institute of Physics.

FANZHANGreceived her B.Sc. and M.Eng. degrees from Bei- jing University of Aeronautics and Astronautics, China, in 1999 and 2002. She is currently a Ph.D. student at the Engineering Department of Cambridge University. Her research interests include storage area networks and optical switching.

MARKFANis currently a research student in the Photonics and Sensors Group, University of Cambridge. He holds an electrical and information science degree from Cambridge, and is actively involved in simulations of packet switches and design and experimental work in realizing VCSEL optical links.

The exact choice of the most suitable switching technology is not

certain and application dependent;

however, what is certain is that both

types of optical switch cores could

solve the next big problem on

the horizon of optical SANs.

Reconfigurable Free-Space

O PTICAL S TORAGE A REA N ETWORKS