from Continuous Distribution Function
Shim, Hyung Jin
Nuclear Engineering Department,
Seoul National University
• Reuven Y. Rubinstein, "Simulation and the Monte Carlo Method," John Wiley &
Sons, Inc., New York (1981).
• C. J. Everett and E. D. Cashwell, "A Third Monte Carlo Sampler" Los Alamos report, LA-9721-MS (1983).
• H. J. Shim, “McCARD Methodology,” SNUMCL/TR001/2010(01), 2012.
References
3.1.2.1 RVG – Principles
(Inverse Transform & Rejection Methods)
From a probability density function (PDF), f(x) for a≤x≤b, the corresponding cumulative probability density function (CDF), F(x) can be defined as
When a random variable X follows a PDF, f(x) and its corresponding CDF, F(x), it can be sampled using a random number, ξ, which is sampled from a uniform
distribution in interval (0,1), as
Proof:
Inverse Transform Method
x d x f x
F
xa
( )
) (
) (
)
(
1
F x X F
( ) ( ) ( )
)
( X x P F
1x P F x F x
P
Explanatory Diagram of Inverse Transform Method
-6 -4 -2 0 2 4 6
0
, 1
2 ; ) exp (
2 ) 1
( 2
2
x xf
< Normal Distribution >
Xe t dt X
erf
erf x x
F
0
2 2
2 ; 2 1
) 1 (
-6 -4 -2 0 2 4 6
1
) (x F
A probability that a particle flies as long as l and collides with a atom can be written as
Then, the flight length can be sampled by
Example #1 – Sampling the flight length
) exp(
)
( l
t tl
p
) ( )
(
) ( )
( )
(
l P l
P
l l
P l
l P l
P
t
t
]
[l P
l l
P[ ] probability that the particle flies as much as ] [l l P
dl l ) exp(
tt x
0
ln )
- ln(1
x
Nuclide t PDF CDF
U-235 0.107 0.216 0.216
U-238 0.211 0.425 0.641
O-16 0.178 0.359 1.000
Example #2 – Selection of a Collided Nuclide
0.000 0.100 0.200 0.300 0.400 0.500
U-235 U-238 O-16
0.000 0.200 0.400 0.600 0.800 1.000 1.200
ki
i t N
i
i t k
i i t
1 1
1 1
It is common that the CDF and its inverse function for a random variable cannot be analytically obtained.
A random variable X, which follows the PDF, f(x) in interval [a,b] can be sampled by trial and error as
① Sample X by using a random number
1.② From another random number
2, accept X if and return to ① elsewhere.Acceptance – Rejection Method – 1/2
) 1
(
a b a X
)
2 f (X
)
2 f (X
)
2 f (X
In order to enhance the sampling efficiency, the PDF f(x) can be represented as
where , is also a PDF, and .
Then X can be sampled as
① Sample X from the PDF of h(x).
② Using a random number ξ, accept X if and reject elsewhere.
Acceptance – Rejection Method – 2/2
) ( ) ( )
( x Ch x g x
f
1
C h ( x ) 0 g ( x ) 1
) ( X
g
The standard normal distribution can be expressed as
Then X can be sampled by
① Sample X from h(x).
② If the below condition is satisfies, accept X. The condition is violated, go to step
①.
Example #1 – Sampling from Normal Distribution
2 exp 1
) ( , )
( 2 ,
; 0
), ( ) 2 (
2 exp )
(
2 2
x x g e
x e h
C
x x
g x x Ch
x f
x
ln( 1 ) ln
0
1
xe
xd x e
xX X
2 ln 1
2 exp 1
2
2
X X
Method 1
phi=2*PI*RNG->GetRN();
sinP=sin(phi); cosP=cos(phi);
Method 2 do {
C1=2.*RNG->GetRN()-1.;
C2=2.*RNG->GetRN()-1.;
C3=C1*C1+C2*C2;
}while(C3>1.);
C4=sqrt(C3); sinP=C1/C4; cosP=C2/C4;
Example #2 – Sampling from Isotropic Distribution
3.1.2.2 Substitution of Variables
in Multiple Integration
Compute the Area of an Ellipse
-2 -1 1 2
-1 -0.5
0.5 1
By substituting x and y as
the double integrals can be computed by
2 2
x y 1
a b
2 2
1
?
x y
a b
A
dxdy
, ,
x y
u v dx adu dy bdv
a b
2 2
2 2
1
1
x y
a b
u v
A dxdy
abdudv ab
y
In the double integrations, we are going to change the original set of variables (x,y) to a set of other variables (u,v).
Then our problem becomes how to convert a small area of A=xy to the corresponding area of A’=uv.
Consider the two-dimensional linear transformation as
Then the rectangular area is converted to the area of a parallelogram as
Integration by Linear Transformation
11 12
21 22
, u a x a y v a x a y
dxdy dudv
From the property of the linear transformation, the scaling factor F in A’=FA is independent of its location and size.
The scaling factor F becomes the area of the parallelogram:
Therefore the integration becomes
Integration by Linear Transformation (Contd.)
11 22 12 21
F A a a a a
dxdy F dudv
1
The area of the parallelogram is the absolute value of the determinant of the matrix formed by the vectors representing the parallelogram's sides.
The volume of the parallelepiped is the absolute value of the determinant of the matrix formed by the rows r1, r2, and r3.
Meaning of Determinant
a b ad bc c d
a b c
e f d f d e
d e f a b c
h i g i g h
g h i
Properties of Determinant
det( A
T) det( ) A det( A
1) 1 det( ) A
det( AB ) det( ) det( ) A B
When the variable x and y are transformed to u and v as
Then, by the linear approximation x and y can be written as
In the same way to the linear transformation, the small rectangular area in x-y coordinate becomes a small area of the corresponding parallelogram in u-v coordinate.
General Transformation
( , ), ( , ) u u x y v v x y
, ;
( or , or )
x y x y
u u x u y v v x v y f
f f u v x y
x y
x y
u u
u x
v v
v y
( ,0) ( , )
(0, ) ( , )
x x
x u x v x
y u y v y
x y
u u
Area x y
v v
In vector calculus, the Jacobian matrix is the matrix of all first-order partial derivatives of a vector- or scalar-valued function with respect to another vector.
Jacobian Matrix
1 1
1
1 1
1
( , , )
( , , )
n
m n
m m
n
F F
x x
F F
J x x
F F
x x
1(
1( )) ( ( )) J
Fp J
Fp
According to the inverse function theorem, the matrix inverse of the Jacobian matrix of an invertible function is the Jacobian matrix of the inverse function.
If m=n, then F is a function from n-space to n- space and the Jacobian matrix is a square
matrix. We can then form its determinant, known as the Jacobian determinant. The Jacobian determinant is sometimes simply called "the Jacobian."
1 1
1
1 1
1
( , , )
( , , )
n
m n
m m
n
F F
x x
F F
J x x
F F
x x
Sometimes, it is often advantageous to evaluate in a coordinate system other than the xy-coordinate system.
The formula for change of variables is given by
where |…| means the absolute value.
Change of Variables in Double Integrals
( , )
R
f x y dxdy
( , )
( , ) ( , ), ( , )
( , )
R S
f x y dxdy f x u v y u v x y dudv u v
Let R be the disc of radius 2 centered at the origin. Calculate
Solution:
Example 1 of Transformation to Polar Coordinates
2 2
sin( ) ?
R
x y dxdy
2 2
cos sin
sin( ) sin
sin cos
R S
x y dxdy r r drd
r
cos , sin x r y r
2 2
2
0r sin r dr
2
2
r t rdr dt
4 0
4 0
sin
cos (1 cos 4) tdt
t
Evaluate
where R is the region between the two circles x2+y2=1 and x2+y2=4.
Solution:
Example 2
2 2
x y R
e
dxdy
2 2 2
2
2 2
0 1
2 2
0 1
1 4
1 2
( )
x y r
R
r
e dxdy e rdrd
e d
e e
The function exp(-x2) has no elementary anti-derivative. But we can evaluate by using the theory of double integrals.
Now transform to polar coordinates x=rcos
, y=rsin
. Hence
Example 3
x2
e dx
2
2
2
2
2
2 2
2
( )
x x x x y
x y
e dx e dx e dx e dx e dy
e dxdy
2
2 22
2
2 ( )
2
0 0
2
0 0
1 2
x x y
r
r
e dx e dxdy
e rdrd
e d
x2
e dx
3.1.2.3 Generations of
Continuous Random Variables
If Z~N(
,
2), its pdf is given bywhere
is the mean and
2 the variance of the distribution. We consider only generation from N(0,1) (standard normal variables), since any random Z~N(
,
2) can be represented as Z=
+
X, where X is from N(0,1). Box and Müller Algorithm:
• Let X and Y be two independent standard normal random variables, so (X,Y) is a random point in the plane.
• Then the pdf of the two random variable, f(x,y) can be expressed as
Normal (Gaussian) Distribution
2 2
1 ( )
( ) exp ,
2 2
f z z
z
2 2
2 2
2 2
( ) 2
1 1
( , )
2 2
1 2
x y
x y
f x y e e
e
• When x=rcos
and y=rsin
, the pdf, f(r,
) becomes• Then r can be sampled by
• Because
can be generated from the uniform distribution over [0,2
], X and Y can be sampled byBox and Müller Algorithm (Contd.)
2/ 2
( cos , sin ) ( , ) ( cos , sin )
( , )
r
r r
f x y dxdy f r r drd
r e rdrd
2/ 2
( , ) ( ) ( );
( ) , ( ) 1
2
R r R
f r f r f f r e r f
2/ 2 2/ 2 2/ 2
0 0
( ) r r r r 1 r
F r
e r dr e e
2/ 2 1
er
2ln 1
r
1 2
2ln cos(2 ),
x
Method 1
phi=2*PI*RNG->GetRN();
sinP=sin(phi); cosP=cos(phi);
Method 2 do {
C1=2.*RNG->GetRN()-1.;
C2=2.*RNG->GetRN()-1.;
C3=C1*C1+C2*C2;
}while(C3>1.);
C4=sqrt(C3); sinP=C1/C4; cosP=C2/C4;
Example #1 – Sampling from Isotropic Distribution
Let’s consider a uniform distribution in a disc of radius of 1:
Then the corresponding pdf for the polar coordinates becomes
Therefore the uniform pdf in a disc can be regarded as the multiplication of fR(r) and f(
) where f(
) follows the isotropic distribution. Then from the sampled X and Y in the xy-coordinates, the R and can be calculated by
Proof of Method 2 for the Isotropic Distribution
( , ) ( ) ( );
( ) 2 , ( ) 1 , [0,1) 2
R
R
f r f r f
f r r f r
1 ( , ) 1
( , )
( , )
f x y dxdy x y drd rdrd
r
2 2
( , ) 1 ; 1
f x y x y
cos , x r
cos
X , sin
Y3.1.2.4 Random Variate Generation
from Joint Distribution
Let x=(x1, , xn)T be a column vector in Rn and A an m×n matrix. The mapping x→z, with z=Ax, is called a linear transformation.
Now consider a random vector X=(X1, , Xn)T, and let Then Z is a random vector in Rm.
Let’s see how the expectation vector and covariance matrix of Z are transformed.
Linear Transformation
Z AX
[ ] [ ] [ ]
E E E
Z X
μ Z AX A X Aμ
[( )( ) ]
[( )( ) ]
[( )( ) ]
T
T
T T
T
E E
E
Z Z Z
X X
X X
X
Σ Z μ Z μ
AX Aμ AX Aμ A X μ X μ A AΣ A
For a linear transformation z=Ax,
For general transformations
, the pdf function for z becomes
Transformation of PDF
( 1 ) ( ) f
f
X Z
z A z
A
1 1
2 2
( )
( ) and ( )
n n( )
x g
x g
x g
x
x z g x
x
1 1
1 1
1 1 1
1 1 1 1
1 1 1
1
( , , )
( ) ( ( )) ( ) ; ( )
( , , )
n
n n
n n
n
g g
z z
g g
f f J J
z z
g g
z z
Z z X g z Z g Z g
Let X~N(0,1). Then X has density fX given by
Now consider the transformation Z=
+
X. Then Z has densityIn other words, Z~N(
,
2). If Z~N(
,
2), then (Z-
) /
~ N(0,1). This procedure is called standardization.Standardization
2
1 2
( ) 2
x
f xX e
2 2 2
1 ( )
( ) exp
2 2
Z
f z x
We now generalize this to n dimensions. Let X1, , Xn be independent and standard normal random variables. The joint pdf of X=(X1, , Xn)T is given by
Consider the affine transformation (that is, a linear transformation plus a constant).
Then the expectation and covariance becomes
Any random vector of the form of is said to have a jointly normal or multivariate normal distribution.
Standardization of Joint PDF
1
1 2
( ) 2
n T
f x e
x x X
Z μ BX
[ ] [ ] [ ] [ ]
E E E E
μZ Z μ BX μ B X μ
[( )( ) ]
[( )( ) ]
[( )( ) ]
[ ]
T
T T
T T
T T
E E E
E
Z Z Z
Σ Z μ Z μ
μ BX μ μ BX μ BX BX
B XX B BIB BB
Z μ BX
Conversely, given a covariance matrix , there exists a unique lower triangular matrix
such that =BBT. This matrix can be obtained efficiently via the Cholesky decomposition.
Jointly Normal Random Variables
11
21 22
1 2
0 0
0
n n nn
b
b b
B
b b b
Let be a covariance matrix. Then we wish to find a matrix B such that =BBT.
The Cholesky square root method computes a lower triangular matrix B via a set of recursive equations as follows:
Cholesky Square Root Method
11
21 22
1 2
0 0
; 0
n n nn
b
b b
B B
b b b
Z μ X
2 2 2
1 11 1 1 [ ]1 11 11 [ ]1
Z b X
Z b b
Z2 2 2
2 21 1 22 2 2 [ ]2 21 22
Z b X b X
Z b b1 2 1 1 2 2
11 1 1 1 21 1 22 2 2 2
11 1 21 1 22 2
11 21
cov[ , ] [( )( )]
[(( ) )(( ) )]
[ ( )]
Z Z E Z Z
E b X b X b X
E b X b X b X b b
2
12 12 , 21
b b
Generally, the bij can be found by
where by convention,
Cholesky Square Root Method (Contd.)
1
1 1
2 1 j
ij ik jk
k
ij j
jj jk
k
b b b
b
0
1
0, 1
ik jk k
b b j i n
