KIAF 유저 가이드

(1)

l Sci

ence

exp

erim

ental D

ate hub C_enter

Global Science experimental Data hub Center

KIAF User Guide

(2)

Preface

¯ 8⌧î KIAF ‹§\ ¨©ê t‹tp, 8⌧ 0Ï⌧ ‹⇣–⌧ • \ ‡X ¥©D Ù‡ à‰. ¨©ê t‹î ¨⌅ı¿ ∆t ¸0 <\ ¥©t î , ≠⌧ t‰.

¨©ê t‹– \ TX∏, 0Ï| X‡ˆ@ Ñ@ ë1ê kong91@kisti. re.kr ò KIAF ¨ê[email protected] <\ }¸‹0 ç »‰.

(3)

3.3 òª⌧ ° ≠⌧ . . . 6 4 ¨©ê ⌧ 7 4.1 ⌧ | . . . 7 4.1.1 ⌧1. Batch test . . . 7 4.1.2 ⌧2. Multi output . . . 9 4.2 ¨©ê Ñ T‹ Ö% )ï . . . 9 4.2.1 Ñ 1. Batch test . . . 9 4.2.2 Ñ 2. Multi output . . . 11 5 ¨©ê Ñ XΩ– 0x î ¨m 12 5.1 Alien token . . . 12 5.1.1 xù⌧ ı¨ . . . 12 5.1.2 Alien Token ⌧ . . . 12

5.1.3 Condor Ã‰x‹– Alien Token ⌧ . . . 12

5.2 PROOF ¨©ï . . . 13

5.2.1 PROOF-Lite ¨©ï . . . 13

(4)

8‡êÃ 17

A GSDC ¨©ê ƒ ‡≠⌧ ë› 18

B KIAF l1 20

(5)

List of Figures

(6)

1 KIAF ⌧Ñ ⌘ç

t –⌧î KIAFX ƒ D ⌧ D KIAF– ⌘çX‡ Ñ – Dî\ AL-ICE XΩD \‹Xî ¸ L¿ LDÙ†µ»‰.

1.1 ¨©ê ƒ

›1

GSDC–⌧ ⌧ıXî KIAF| t©X0 ⌅t⌧î KIAF ƒ D ⌧ D| i»‰. Ï0–⌧î KIAF ƒ D ⌧ î ¸ D H¥i»‰.

1.1.1 ƒ ‡≠⌧

ƒ ‡≠⌧î DòX ¯ºtò Ä]X ƒ ‡≠⌧ ë›D ¨©XÏ ë1i »‰.

(ƒ ‡≠⌧î ƒ ⌧ tƒ ‡≠⌧î â‹ –0 )»‰.)

GSDC User Account Subscription Form User’s Information

Account Name VO (Research Group) First Name (Native) Last Name (Native) First Name (English) Last Name (English) Position Country Organization (Full Name) Native English Email Phone Supervisor’s Information

First Name (Native) Last Name (Native) First Name (English) Last Name (English) Organization Position Email Phone * Please, acquire Supervisor hand-written signature.

Subscriber’s Signature : ( date : . . . ) Supervisor’s Signature : ( date : . . . ) GSDC Director’s Signature : ( date : . . . )

Figure 1: GSDC ¨©ê ƒ ‡≠⌧ ë›

Account Name@ ¨©ê ID ÄÑ<\ LDÙ0 }å ë11_{X‡, ƒ t} ›1⌧ ƒ Ñ‹ D à8 ¨©ê Email\ ⌅° )»‰.

1_ID| _{UXå Ã‰0 ⌅t tƒ ‡≠⌧| Ù˘ê–å ⌧ú `L T|– ⇡t 0ÖXÏ Ù¥0|}

(7)

ë1\ ‡≠⌧î KISTI KIAF Ù˘ê–å ⌧úi»‰. ƒ î≠ Ù˘ê : [email protected]

1.2 UI ⌧Ñ ⌘ç

1.3 ALICE XΩ \)

kiafenv Ö9¥| ¨©XÏ KIAF– Dî\ ALICE XΩD \‹i»‰.

[kong91@kiaf-test-01 ˜]$ kiafenv Load the ALICE environment

tL \‹⇠î XΩX AliRoot 9@ AliPhysicsX Ñ<@ $HOME/.alice env/alice env.conf –⌧ X)»‰.

[kong91@kiaf-test-01 ˜]$ cat $HOME/.alice_env/alice_env.conf export ALIPHYSICS_VERSION="vAN-20160417-1"

(8)

export ALICE_ROOT_VERSION="v5-07-20-4"

KIAF–⌧ ¨© •\ AliRoot ✏ AliPhysicsX Ñ<@ http://alimonitor. cern.ch/packages–⌧ Uxt •i»‰.¨©X‡ê Xî Ñ<X CVMFS

status Availablex¿ Ux X‹0 ç»‰.¨©t à •\ Ñ<D ‰â` Ω

∞ –Ï ⌧›Xp AliRoot ‰â ⇠¿ Jµ»‰.

kiafenv Ö9¥X 5X<\ -t ⇣î --token¸ -p ⇣î --pod t¨X p, t5X@ Alien tokenD ⌧ X‡ p5X@ PoDXΩD \‹i»‰. -h ⇣î --help| Ö%‹ ƒ¿–D ú%i»‰. ®‡ 5X@ Ÿ‹– ¨©t •i»‰. 5XX token¸ PoD– t⌧î 5.1¸ 5.2–⌧ $Öi»‰.

[kong91@kiaf-test-01 ˜]$ kiafenv -h

This command load the ALICE environment for KIAF options:

-p, --pod : Load the PoD environment

-t, --token : Create alien session token

(9)

2 pt0 K

t –⌧î pt0KD î≠X‡ Ux Xî )ï– t LDÙ†µ»‰.

2.1 pt0 K î≠

KIAF–⌧î Ñ – Dî\ pt0| \Ï\ .® ì@§ Ñ D ⇠âi» ‰. tL Dî\ pt0 KD Ù˘ê–å î≠ Xƒ] i»‰. î≠@ T|D µ t Ù˘ê–å Dî\ pt0KX Ù| Ù¥¸‹t )»‰. pt0KX Ùî Data:Sim, Period, Run Number, Variant, Pass Ò Ö»‰. î≠X‡ pt0 K@ /xrootfs/alice Dò– Alien Catalogue@ ⇡@ Ω\\ • ⇠¥ àµ»‰.

- /xrootdfs/alice/alice/sim/2013/LHC13d3/195675/AOD159/0001/root archive.zip pt0K î≠ Ù˘ê : [email protected] ⌅ T|@ pt0KD î≠` L–Ã t©X‡, |⇠ x 8Xî kiaf-support@ kisti.re.kr| t©X0| î≠ ‹Ω»‰.

2.2 pt0 K Ux

⌅–⌧ ∏ \ É¸ ⇡t KIAF– •⇠¥ àî pt0î /xrootdfs/alice Dò – ⌅Xi»‰. ¡⌘  †¨– ⌘¸XÏ |D Ux `⇠ àµ»‰. ⇣\ î≠\ pt0 KX | ¨§∏î /pool/datalist H– type period run pass variant.txt X ›<\ t¨i»‰. - sim LHC13d3 195675 AOD159.txt ⇣\ Ö9¥showdatasetD µt §Lt’ ⌅iD L ⇠ àµ»‰. Ö9¥@ hÿ Ä…D –Xî pt0 KX tÑD ¥¸t t˘Xî pt0KÃ ÙÏ » ‰. pt0KD ¥¸¿ J<t ⌅¥| påXp, på⇠î ë– 0| Œ@ ‹⌅t åî ⇠ àµ»‰.

[kong91@kiaf-test-01 ˜]$ showdatalist LHC12i

Dataset Queued Downloading Success Fail

data/2012/LHC12i/000192762/ESDs/muon_calo_pass2 0 0 220 0

data/2012/LHC12i/000192765/ESDs/muon_calo_pass2 0 0 480 0

(10)

3 HTCondor ¨©ï

t –⌧î Ñ – ¨© ⇠î HTCondorX 0¯ x ¨©ï– t å⌧X‡ ê i»‰.

3.1 Condor cluster ¡‹ Ux

condor statusî ⌅¨ condor ¨oX ¡iD ÙÏ¸î Ö9¥Ö»‰.

[kong91@kiaf-test-01 ˜]$ condor_status

Name OpSys Arch State Activity LoadAv Mem ActvtyTime

slot10@kiaf-test-0 LINUX X86_64 Unclaimed Idle 0.000 3225 3+06:15:57

...

Machines Owner Claimed Unclaimed Matched Preempting

X86_64/LINUX 160 0 4 156 0 0

Total 160 0 4 156 0 0

3.2 Condor job Queue Ux

condor q Ö9¥î î≠ \ °X queue| ÙÏ »‰. °t ⇠â ⌘– Iò HX ¡ \ ¿Xt \¯| Ux XÏ òª ⌧ ÄÑD ⇠ t| i»‰. °X ¡‹ –Ã D»| ID, OWNER Òƒ Uxt • i»‰.

[kong91@kiaf-test-01 multi_output]$ condor_q

-- Schedd: kiaf-test-01.sdfarm.kr : <134.75.125.139:14337?...

ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD

9618.0 kong91 6/13 16:58 0+00:00:08 R 0 0.3 condor_dagman -p 0

9619.0 kong91 6/13 16:59 0+00:00:00 I 0 0.0 job_wrap.sh AliPhy

(11)

6 jobs; 0 completed, 0 removed, 5 idle, 1 running, 0 held, 0 suspended

3.3 òª⌧ ° ≠⌧

ë≈– 8⌧ à¥ °t ⌘¿⇠»DL ⌧\ °D ≠⌧Xî Ö9¥Ö»‰. condor rm ID| Ö% XÏ ⇠â i»‰.

(12)

4 ¨©ê

⌧

»må lï⌧ KIAFî HTCondor|î 0X ‹§\D t©i»‰. t –⌧ î ⌧ T‹ ¨©ï¸ ⌧ T‹| t©\ ¨©ê Ñ )ï– t LD Ù†µ »‰.

4.1 ⌧

|

4.1.1 ⌧1. Batch test

⌧ |@ /tutorial Dò– àµ»‰. batch test ÙT| ı¨Xpò, batch test.tar |D ı¨ \§ UïD Ä¥ ¨©Xt )»‰. cp -r /tutorial/batch_test $HOME/batch_test ⇣î cp -r /tutorial/batch_test.tar $HOME/batch_test.tar tar -xvf batch_test.tar batch test ÙT HX | ¨§∏î Dò@ ⇡µ»‰. [kong91@kiaf-test-01 batch_test]$ ls -l total 28

-rw-r--r--. 1 kong91 kong91 776 Jun 7 10:06 AddMyTask.C

-rw-r--r--. 1 kong91 kong91 2036 Jun 7 10:06 AliAnalysisTaskMyTask.cxx

-rw-r--r--. 1 kong91 kong91 895 Jun 7 10:06 AliAnalysisTaskMyTask.h

-rw-r--r--. 1 kong91 kong91 1120 Jun 7 10:06 runAnalysis.C

-rwxrw-r--. 1 kong91 kong91 3546 Jun 7 12:56 runCondor.sh

-rw-rw-r--. 1 kong91 kong91 5325 Jun 7 10:06 sim_LHC13d3_195675_AOD159.txt

(13)

‰â tƒ –Ï∆t ‰ât • i»‰.

tL runCondor.shî runAnalysis.CX xã |D ëå ò⌅¥ —, ƒ∞

D ƒ@¸î ‰l\ Ö»‰. â, runAnalysis.C ‰l\ ê¥X –Ï ⌧›\‰t

runCondor.sh–⌧ƒ –Ï ⌧›X» local test| < ⇠âX8 å•i»‰.

./runCondor.sh Ö9D µt §lΩ∏| ‰âXt Dò@ ⇡@ ∞¸| 0¯⇣D µt ¸ ⇠ àµ»‰.

[kong91@kiaf-test-01 batch_test]$ ./runCondor.sh

Renaming rescue DAGs newer than number 0

---File for submitting this DAG to Condor : 160612173845/macro/condor.dag.condor.sub

Log of DAGMan debugging messages : 160612173845/macro/condor.dag.dagman.out

Log of Condor library output : 160612173845/macro/condor.dag.lib.out

Log of Condor library error messages : 160612173845/macro/condor.dag.lib.err

Log of the life of condor_dagman itself : 160612173845/macro/condor.dag.dagman.log

Submitting job(s).

1 job(s) submitted to cluster 9244.

---tL 160612173845î °D ⇠â‹® D‘|‹Ñ X tÑ\ Ã‰¥ƒ ÙT t p, Ãπ  †¨\ ¨©)»‰. °X \¯î Ãπ †¨ Dò logsÙT– ⌅Xi»‰. \Ö ∞¸ |@ batch testÙT H– merge AnalysisResults.root |î tÑ<\ ›1 )»‰.

⇠â⌘x °X Pîcondor q |î Ö9¥| µt Uxt • i»‰.

[kong91@kiaf-test-01 batch_test]$ condor_q

9317.0 kong91 6/13 00:15 0+00:00:08 R 0 0.3 condor_dagman -p 0

9318.0 kong91 6/13 00:15 0+00:00:01 R 0 0.0 job_wrapper.sh Ali

(14)

4.1.2 ⌧2. Multi output

tà–î Ñ ∞¸\ ÏÏ⌧X D√ã |t ›1 ⇠î ⌧ Ö»‰.

⌧ |@ /tutorial Dò– àµ»‰. multi output ÙT| ı¨Xpò, multi output.tar |D ı¨ \§ UïD Ä¥ ¨©Xt )»‰. ⌅X ⌧@ ⇡t l1 ⇠¥ à

µ»‰. ./runCondor.sh Ö9D µt §lΩ∏| ‰âXt 2⌧X ∞¸ |t ›1 ⇠î Ét Ux • i»‰.

[kong91@kiaf-test-01 multi_output]$ ls -ls *root

26 -rw-r--r--. 1 kong91 kong91 4265 Jun 13 16:34 merge0_AnalysisResults.root 26 -rw-r--r--. 1 kong91 kong91 4265 Jun 13 16:34 merge1_AnalysisResults.root

(15)

run_macro="runAnalysis.C"

data_file="sim_LHC13d3_195675_AOD159.txt"

input_files="AddMyTask.C,AliAnalysisTaskMyTask.cxx,AliAnalysisTaskMyTask.h" out_file="AnalysisResults.root"

• condor slot condor– Ÿ‹– `˘ ¨oX \ ✓ Ö»‰. ⌅¨ KIAF –î 160⌧X ¨ot D ⇠¥ à<¿\ ‰x ¨©ê‰X –`\ ¨©D ⌅XÏ 40X ✓D å• i»‰.

• nfile ¨o˘ Ñ0 ⇠î xã |X /⇠ Ö»‰. Grid°–⌧ CreateAlien-Handler.CX SetSplitMaxInputFileNumberX ✓¸ Ÿ|\ Ì`D i»‰. cat data file | wc -l Ö9¥2_{| µt ⌅¥ xã} _{|X /⇠| L⇠} à<», t| † \ ˘\ ✓D ¨©ê 0Ö i»‰.

• work dir ›1 ⇠î Ãπ  †¨X tÑ<\ ¨©ê –Xî tÑ<\ ¿ΩXÏƒ 4)i»‰.

• run macro aliroot| µt ‰â ⇠î Ñ ‰l\ Ö»‰.

• out file ∞¸| µt ªî root |X tÑ<\ ⌅¨î 1⌧X ∞¸ | Ã ›1t • i»‰. \Ö ∞¸ |@ runCondor.sh@ ⇡@ ⌅X– merge out fileÖ<\ ›1 )»‰.

(16)

4.2.2 Ñ 2. Multi output tà–î Ñ ∞¸\ ÏÏ⌧X D√ã |t ›1 ⇠î Ω∞| ⌅\ ‰l\ Ö»‰. ⌅ ⌧@ ⇡t L§∏ DÃ⌧ ‰l\| D i»‰. ‰l\ D ⇠»DL ⇠ t|` |@ runCondor.sh |Ö»‰. runCondor.sh |X ⌫ÄÑ@ Dò@ ⇡µ»‰. condor_slot=40 nfile=10 work_dir=‘date +%y%m%d%H%M%S‘ run_macro="runAnalysis.C" data_file="sim_LHC13d3_195675_AOD159.txt" input_files="AddMyTask.C,AliAnalysisTaskMyTask.cxx,AliAnalysisTaskMyTask.h" out_files="AnalysisResults.root, AnalysisResults.root" Ñ 1¸ Ï|ƒ ÄÑ@ out filesÖ»‰. tÛ– ›1 D√ã |X tÑD d»(,)\ lÑ XÏ ¥¸t, \Ö ∞¸ |@ runCondor.sh@ ⇡@ ⌅X– merge# out fileÖ3<\ ›1 )»‰.

(17)

5 ¨©ê Ñ

XΩ– 0x î ¨m

5.1 Alien token

5.1.1 xù⌧ ı¨

$HOME/.globus  †¨– ⌧x xù⌧ |x usercert.pem, userkey.pem D ı¨i»‰.

⌧x 8‡–⌧scp Ö9¥| ¨©XÏ ı¨ Xt)»‰.

kong91:˜/> scp -r -P 4280 $HOME/.globus [email protected]:/alice/home/kong91/. [email protected]’s password:

usercert.pem 100% 3334 3.3KB/s 00:00

userkey.pem 100% 1907 1.9KB/s 00:00

5.1.2 Alien Token ⌧

†p@alien-token-init Ö9¥| µt ⌧ t •i»‰. ‰Ã KIAF ƒ Ö¸ CERN ƒ Öt |XX¿ Jî Ω∞, alien-token-init §– CERN ƒ ÖD ¥ ⇠| i»‰.

⇣\kiafenv -t | µ\ †p ⌧ D ⌅t $HOME/.alice env/alice env.conf – \⌅D î XÏ, export alien API USER=CERN ƒ ÖD ¥¸‹0 ç »‰.

5.1.3 Condor Ã‰x‹– Alien Token ⌧

(18)

5.2 PROOF ¨©ï

PROOF-Liteî πƒ\ $ ∆t AliROOTX Ö9D t©t ‰ât • i» ‰.(PROOF-Liteî ⌧⌅¥ DÃ UI8‡X ®‡ CPU| ¨©i»‰.) AliROOT | ‰â\ ƒ Proof 8XD $ i»‰.

[kong91@kiaf-test-01 ˜]$ aliroot -l root [0] TProof::Open("")

+++ Starting PROOF-Lite with 40 workers +++ Opening connections to workers: OK (40 workers) Setting up worker servers: OK (40 workers) PROOF set to parallel mode (40 workers) (class TProof*)0x2d20420 root [1] UIX CPU 40⌧| ®P ¨©XÏ 40⌧X Ã‰ D⇠»LD Ux ` ⇠ àµ »‰. 5.2.2 PoD ¨©ï PoDî Proof| ¨©ê –Xî \ $ XÏ ¨©Xƒ] ƒ@¸î ¥ ¨ tX Ö»‰. PoD| ¨©Xt ⌧ ⌅¥X CPUê–D \© ` ⇠ àµ»‰. PoD XΩ $

aliroot| ‹ë`L warning 8l ò¿ † ⇠ àµ»‰.)

[kong91@kiaf-test-01 ˜]$ kiafenv -p Load the ALICE environment

(19)

Generating a default PoD configuration file... Generating a default PoD configuration file - DONE.

⌅ Ö9D ⇠âXt ¨©ê  †¨ Dò– .PoDX ÙT ›1 )»‰. ÙT ¥ X PoD.cfg |D ⇠ XÏ 8Ä p t • i»‰. |D ⇠ \ §–î podX ΩD ‰‹ àÏ@| i»‰. tLî source /pool/PoD/3.16/PoD env.sh Ö9D ¨©i»‰.

PoD ‰â )ï

[kong91@kiaf-test-01 ˜]$ pod-server start Starting PoD server...

updating xproofd configuration file... starting xproofd...

starting PoD agent...

preparing PoD worker package...

select user defined environment script to be added to worker package... selecting pre-compiled bins to be added to worker package...

PoD worker package: /share/kong91/.PoD/wrk/PoDWorker.sh

---XPROOFD [2860349] port: 21002 PoD agent [2860374] port: 22002

PROOF connection string: [email protected]:21002

(20)

-r 5X@ PROOF ¨©` 0X‹§\D ¿ Xî ÉÖ»‰. KIAF–⌧î condor Ã ¿– X0 L8– t ✓@ ¿Ω` ⇠ ∆µ»‰. -n 5X@ ‰â` PROOF Ã‰X /⇠Ö»‰. condor| ¨©X0 L8– \ /⇠î 160⌧ Ö»‰. ⇣\ PROOF| condor| µt ‰â X0 L8– condorX ° ID| `˘ µ»‰.

PoD D ⇠»î¿îpod-info -n<\ Uxi»‰. D DÃ⌧ PROOF X /⇠| \0i»‰. condor| ‰âXîp }⌅X ‹⌅t Dîh<\ ¿ \ / ⇠@ ﬁ¿ Jî Ω∞ †‹ 0‰Ω»‰.

[kong91@kiaf-test-01 ˜]$ pod-info -n 10

condor Ö9¥x condor q| µt⌧ƒ Ux ` ⇠ àµ»‰. PoD Ã‰ ‰ ât •Xt condorX ¡‹ R\ \‹ )»‰.

[kong91@kiaf-test-01 ˜]$ condor_q

(21)

Starting master: opening connection ... Starting master: OK

Note: File "iostream" already loaded PROOF set to sequential mode

(22)

8‡

êÃ

[1] HTCondor Ö8 (2016 GSDC School of Data Grid Computing). https://indico.cern.ch/event/476290/contributions/ 1154545/attachments/1221990/1786975/HTCondor___.pdf. [2] PROOF tutorial. https://root.cern.ch/root/htmldoc/

tutorials/proof/index.html.

(23)

A

GSDC ¨©ê ƒ

‡≠⌧ ë›

(ë1 ⌧⌧– 0| ⌧ D ‹ƒ i»‰.)

• Position ⌅¨ ¨©êX ¡⌅| Ö%i»‰. ( ¨¸ Y›, ¨¸ Y›, ¨ƒ l– Ò) • Country ⌅¨ åç⌧ 0 t ⌅X\ m | Ö%i»‰. • Organization ⌅¨ åç⌧ 0 D 0Öi»‰.

(24)

GSDC User Account Subscription Form

User’s Information

Account Name VO (Research Group)

First Name (Native) Last Name (Native)

First Name (English) Last Name (English)

Position Country Organization (Full Name) Native English Email Phone

Supervisor’s Information

First Name (Native) Last Name (Native)

First Name (English) Last Name (English)

Organization Position

Email Phone

* Please, acquire Supervisor hand-written signature.

Subscriber’s Signature : ( date : . . . )

Supervisor’s Signature : ( date : . . . )

(25)

(26)

(27)

KIAF 유저 가이드

Global Science experimental Data hub Center

KIAF User Guide

Preface

Contents

List of Figures

1

KIAF ⌧Ñ ⌘ç

1.1

¨©ê ƒ

›1

1.2

UI ⌧Ñ ⌘ç

1.3

ALICE XΩ \)

2

pt0 K

2.1

pt0 K î≠

2.2

pt0 K Ux

3

HTCondor ¨©ï

3.1

Condor cluster ¡‹ Ux

3.2

Condor job Queue Ux

3.3

òª⌧ ° ≠⌧

4

¨©ê

⌧

4.1

⌧

|

5

¨©ê Ñ