Interpolating Polarization From DoFP Using Stokes Vector
2023-7-4 10:33:26
Polarization plays an important role in both industry and research. By using the different polarization properties, we could eliminate the highlight on the object’s surface, or detect the stress on it1. Some polarized camera sensors are manufactured, to cater for those applications. For example, division of time(DoT), division of amplitude(DoAM), division of aperture(DoAP) and recently, division of focal plane polarimeters(DoFP)2.
DoFP imaging sensors are compact and non-moving, which could capture polarized images in a snapshot. Sony IMX250MYR CMOS sensors are well-designed DoFP devices, which embed on-chip polarizers of different polarization angles for each pixel(Figure), but not on-glass polarizers, to reduce the crosstalk of polarized light.
In clockwise, the polarizers on IMX250MYR CMOS from left bottom to right bottom pixels are 13 5 ∘ 135^\circ 135∘, 9 0 ∘ 90^\circ 90∘, 4 5 ∘ 45^\circ 45∘, 0 ∘ 0^\circ 0∘, and four polarized photodiodes constitute a colour sensor module and four colour sensor modules constitute a colour pixel, which obeys the Bayer rule, is RGGB. After one shot using IMX250MYR, we could get a full-resolution Bayer image, and each Bayer pixel consists of four polarized responses, hence we could decompose the Bayer image into four half-resolution polarized images of each polarization angle. In recent years, many researchers worked so hard, aim to solve the problem of demosaicking the polarized sensor, because they want to fully use the resolution of the DoFP, but the response intensity of no-polarization at each pixel position is difficult to estimate.3 If we just use the polarized half-resolution image, we may not need to care about the demosaicking problem. However, in many scenarios, we may need to generate a virtual polarization filter. Before illustrating how to do that, we should know about the Stokes vector.
Stokes vector with Stokes parameters is a set of values that describe the polarization state(Wiki). The Stokes vector S S S can be defined as S ( I , Q , U , V ) S(I, Q, U, V) S(I,Q,U,V), where I I I is the total intensity of four polarized responses, Q Q Q and U U U present the linear polarization, while V V V presents the circular polarization state. In this case we only consider the linear polarization:
S = [ I Q C ] = [ S 0 S 1 S 2 ] = [ 1 2 ( I 0 + I 45 + I 90 + I 135 ) I 0 − I 90 I 45 − I 135 ] = [ I t o t I t o t P cos ( 2 ϕ ) I t o t P sin ( 2 ϕ ) ] \begin {align*} & S = \begin{bmatrix} I\\ Q\\ C \end{bmatrix} =\begin{bmatrix} S_0\\ S_1\\ S_2 \end{bmatrix} =\begin{bmatrix} \frac{1}{2} (I_0+I_{45}+I_{90}+I_{135})\\ I_0-I_{90}\\ I_{45}-I_{135} \end{bmatrix} =\begin{bmatrix} I_{tot}\\ I_{tot}\mathcal{P}\cos(2\phi)\\ I_{tot}\mathcal{P}\sin(2\phi) \end{bmatrix} \end {align*} S= ⎨ ⎧M0=21 110110000 ,M45=21 101000101 ,M90=21 1−10−110000 M135=21 10−1000−101
However, this is only in the ideal occasion. In such conditions, the cross-polarized light could be completely blocked. In reality, the polarizers usually are imperfect due to manufacture. The parallel and perpendicular transmittance become k 1 k_1 k1 and k 2 k_2 k2, then diattenuation is D = ( k 1 + k 2 ) / ( k 1 − k 2 ) D=(k_1+k_2)/(k_1-k_2) D=(k1+k2)/(k1−k2), and extinction ratio is E R = k 1 / k 2 = ( D + 1 ) / ( D − 1 ) ER=k_1/k_2=(D+1)/(D-1) ER=k1/k2=(D+1)/(D−1)3, and the imperfect Müller matrix become:
M θ = 1 2 [ k 1 + k 2 ( k 1 − k 2 ) cos ( 2 θ ) ( k 1 − k 2 ) sin ( 2 θ ) ( k 1 − k 2 ) cos ( 2 θ ) ( k 1 + k 2 ) cos 2 ( 2 θ ) + 2 k 1 k 2 sin 2 ( 2 θ ) ( k 1 + k 2 − 2 k 1 k 2 ) sin ( 2 θ ) cos ( 2 θ ) ( k 1 − k 2 ) sin ( 2 θ ) ( k 1 + k 2 − 2 k 1 k 2 ) sin ( 2 θ ) cos ( 2 θ ) ( k 1 + k 2 ) sin 2 ( 2 θ ) + 2 k 1 k 2 cos 2 ( 2 θ ) ] \begin{align*} & M_{\theta}=\frac{1}{2} \begin{bmatrix} k_1+k_2& (k_1-k_2)\cos(2\theta)& (k_1-k_2)\sin(2\theta)\\ (k_1-k_2)\cos(2\theta)& (k_1+k_2)\cos^2(2\theta)+2\sqrt{k_1k_2}\sin^2(2\theta)& (k_1+k_2-2\sqrt{k_1k_2})\sin(2\theta)\cos(2\theta)\\ (k_1-k_2)\sin(2\theta)& (k_1+k_2-2\sqrt{k_1k_2})\sin(2\theta)\cos(2\theta)& (k_1+k_2)\sin^2(2\theta)+2\sqrt{k_1k_2}\cos^2(2\theta) \end{bmatrix} \end{align*} Mθ=21 k1+k2(k1−k2)cos(2θ)(k1−k2)sin(2θ)(k1−k2)cos(2θ)(k1+k2)cos2(2θ)+2k1k2sin2(2θ)(k1+k2−2k1k2)sin(2θ)cos(2θ)(k1−k2)sin(2θ)(k1+k2−2k1k2)sin(2θ)cos(2θ)(k1+k2)sin2(2θ)+2k1k2cos2(2θ)
Code Example:
def PolarInterp_Stokes(angle):
anlge_rec = angle/180 * np.pi
height = polar_0.shape[0]
width = polar_0.shape[1]
polar_0_vec = np.double(np.reshape(polar_0, (1, height*width)))
polar_45_vec = np.double(np.reshape(polar_45, (1, height*width)))
polar_90_vec = np.double(np.reshape(polar_90, (1, height*width)))
polar_135_vec = np.double(np.reshape(polar_135, (1, height*width)))
S = np.zeros((3, height*width))
S[0, :] = (1/2)*(polar_0_vec + polar_45_vec + polar_90_vec + polar_135_vec)
S[1, :] = polar_0_vec - polar_90_vec
S[2, :] = polar_45_vec - polar_135_vec
M_angle = np.zeros((3, 3))
M_angle[0, 0] = 1;
M_angle[0, 1] = np.cos(2*anlge_rec)
M_angle[0, 2] = np.sin(2*anlge_rec)
M_angle[1, 0] = np.cos(2*anlge_rec)
M_angle[1, 1] = np.cos(2*anlge_rec) * np.cos(2*anlge_rec)
M_angle[1, 2] = np.sin(2*anlge_rec) * np.cos(2*anlge_rec)
M_angle[2, 0] = np.sin(2*anlge_rec)
M_angle[2, 1] = np.sin(2*anlge_rec) * np.cos(2*anlge_rec)
M_angle[2, 2] = np.sin(2*anlge_rec) * np.sin(2*anlge_rec)
M_angle = (1/2)*M_angle
S_angle = np.double(np.matmul(M_angle, S))
S_angle[0,:][S_angle[0,:]<0] = 0
S_angle[0,:][S_angle[0,:]>255] = 255
Image_angle = np.uint8(np.reshape(S_angle[0, :], (height, width)))
return Image_angle
Result(Cross polarization, Polarization angle from 0~180):
References
chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://dce9ugryut4ao.cloudfront.net/LUCID-Going-Polarized-White-Paper.pdf ↩︎
Zhang J, Luo H, Hui B, et al. Image interpolation for division of focal plane polarimeters with intensity correlation[J]. Optics express, 2016, 24(18): 20799-20807. ↩︎
Qiu S, Fu Q, Wang C, et al. Linear polarization demosaicking for monochrome and colour polarization focal plane arrays[C]//Computer Graphics Forum. 2021, 40(6): 77-89. ↩︎ ↩︎