Infrared and visible image fusion via saliency analysis and local edge-preserving multi-scale decomposition

Xiaoye Zhang; Yong Ma; Fan Fan; Ying Zhang; Jun Huang

doi:10.1364/JOSAA.34.001400

Infrared and visible image fusion via saliency analysis and local edge-preserving multi-scale decomposition

Xiaoye Zhang, Yong Ma, Fan Fan, Ying Zhang, and Jun Huang

Journal of the Optical Society of America A
Vol. 34,
Issue 8,
pp. 1400-1410
(2017)
•https://doi.org/10.1364/JOSAA.34.001400

Get Citation
Copy Citation Text
Xiaoye Zhang, Yong Ma, Fan Fan, Ying Zhang, and Jun Huang, "Infrared and visible image fusion via saliency analysis and local edge-preserving multi-scale decomposition," J. Opt. Soc. Am. A 34, 1400-1410 (2017)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Get PDF (4746 KB)
Set citation alerts
Save article
Editors' Pick

Related Topics
Table of Contents Category
- Image Processing
Optics & Photonics Topics
?

The topics in this list come from the OSA Optics and Photonics Topics applied to this article.
Previously assigned OCIS codes
- Image processing (100.0100)
- Digital image processing (100.2000)
- Fusion (350.2660)

About this Article
History
- Original Manuscript: April 7, 2017
- Revised Manuscript: June 26, 2017
- Manuscript Accepted: June 29, 2017
- Published: July 21, 2017

Not Accessible

Your account may give you access

Abstract

To retain the details of a visible image with a discernible target area, we propose a multi-scale decomposition image fusion method based on a local edge-preserving (LEP) filter and saliency detection. We first use a LEP filter to decompose the infrared and visible images. Then, a modified saliency detection method is utilized to detect the salient target areas of an infrared image, which determine the base layer’s weights of fusion strategy. Finally, each layer is reconstructed to obtain a visually pleasing fused image. Comparison with 11 other state-of-the-art methods reveals the superiority of the proposed method in terms of quality and quantity results.

Full Article | PDF Article

OSA Recommended Articles

Infrared and visible image fusion with spectral graph wavelet transform

Xiang Yan, Hanlin Qin, Jia Li, Huixin Zhou, and Jing-guo Zong
J. Opt. Soc. Am. A 32(9) 1643-1652 (2015)

Infrared and visible image perceptive fusion through multi-level Gaussian curvature filtering image decomposition

Wei Tan, Huixin Zhou, Jiangluqi Song, Huan Li, Yue Yu, and Juan Du
Appl. Opt. 58(12) 3064-3073 (2019)

Infrared and visible image fusion based on total variation and augmented Lagrangian

Hanqi Guo, Yong Ma, Xiaoguang Mei, and Jiayi Ma
J. Opt. Soc. Am. A 34(11) 1961-1968 (2017)

References

You do not have subscription access to this journal. Citation lists with outbound citation links are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access OSA Member Subscription

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access OSA Member Subscription

Figures (7)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access OSA Member Subscription

Tables (2)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access OSA Member Subscription

Equations (20)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access OSA Member Subscription

Metrics

You do not have subscription access to this journal. Article level metrics are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access OSA Member Subscription

Scenes	Indexes	Proposed	CVT	DTCWT	GFF	LP	LPSR	MSVD	Wavelet	WLS	NSCT	HMSD	PC-GF
2 men in front of house	EN	6.9632	6.6638	6.6546	7.2518	6.7831	7.1301	6.4700	6.4261	6.5106	6.6949	7.0773	6.4121
	SF	7.8831	5.7364	5.7463	5.6928	5.8793	5.8303	5.3659	3.6895	5.1383	5.8504	6.4003	4.2342
	SD	53.135	21.917	27.434	44.620	30.155	52.146	23.850	23.211	24.479	27.968	45.914	21.986
	NMI	0.5860	0.6692	0.6747	0.6553	0.6794	0.6744	0.3461	0.4387	0.4801	0.4539	0.4329	0.4407
	${MSSIM}_{F}^{V}$	0.8835	0.8724	0.8789	0.9698	0.8724	0.8976	0.8294	0.8270	0.8995	0.8606	0.8718	0.7499
	${MSSIM}_{F}^{I}$	0.3726	0.5750	0.5767	0.4568	0.5891	0.5535	0.6951	0.7470	0.6048	0.6124	0.5570	0.7877
	${MSSIM}_{F}^{V, I}$	0.6281	0.7237	0.7278	0.7133	0.7308	0.7256	0.7623	0.7870	0.7521	0.7365	0.7144	0.7688
Bench	EN	7.4699	6.9685	6.9390	7.5215	7.1144	7.4337	6.6913	6.5319	6.8030	6.9607	7.3898	6.7532
	SF	12.344	10.611	10.599	10.585	10.650	10.658	10.059	7.4912	9.7313	10.600	11.034	8.2173
	SD	30.385	28.659	32.242	35.196	60.080	33.297	48.063	27.158	27.180	44.111	29.330	37.280
	NMI	0.3934	0.4604	0.5077	0.5064	0.4655	0.4594	0.5132	0.4717	0.4850	0.5905	0.5117	0.5157
	${MSSIM}_{F}^{V}$	0.7651	0.7062	0.8333	0.8445	0.8829	0.7169	0.8820	0.9119	0.9172	0.9896	0.9199	0.9106
	${MSSIM}_{F}^{I}$	0.3761	0.5205	0.3061	0.2501	0.2297	0.5121	0.3651	0.5708	0.5701	0.4742	0.5672	0.5600
	${MSSIM}_{F}^{V, I}$	0.4807	0.5387	0.5391	0.5641	0.5455	0.5459	0.5706	0.6134	0.5697	0.5473	0.5563	0.6145
Lake	EN	7.4016	6.6741	6.6578	7.3687	6.7550	7.1976	6.5683	6.5328	6.6130	6.6852	7.1569	6.5045
	SF	8.1626	5.6114	5.5899	5.7126	5.6665	5.7088	5.0750	3.1921	4.9603	5.6369	6.3033	3.5541
	SD	48.063	27.158	27.180	44.111	29.330	37.280	24.684	24.101	25.763	27.381	37.926	25.096
	NMI	0.5132	0.4717	0.4850	0.5905	0.5117	0.5157	0.3724	0.4481	0.5094	0.4802	0.4696	0.4355
	${MSSIM}_{F}^{V}$	0.8820	0.9119	0.9172	0.9896	0.9199	0.9106	0.8584	0.8410	0.9306	0.9100	0.9171	0.7904
	${MSSIM}_{F}^{I}$	0.3651	0.5708	0.5701	0.4742	0.5672	0.5600	0.7021	0.7731	0.6036	0.5889	0.5304	0.7913
	${MSSIM}_{F}^{V, I}$	0.6236	0.7414	0.7437	0.7319	0.7435	0.7353	0.7803	0.8071	0.7671	0.7494	0.7238	0.7908
Sandpath	EN	7.2040	6.4548	6.4059	6.8402	6.5898	7.1107	6.1616	6.0840	6.6190	6.4548	7.1324	6.4192
	SF	9.8059	7.0677	6.9077	6.6894	7.7551	7.7702	6.8572	4.8114	5.6727	7.4500	8.6213	5.8087
	SD	40.987	23.082	22.405	33.654	25.627	39.371	19.201	18.458	19.403	23.016	39.152	24.752
	NMI	0.5029	0.4030	0.4313	0.4982	0.4479	0.4551	0.3493	0.4085	0.4185	0.4384	0.4028	0.4380
	${MSSIM}_{F}^{V}$	0.9012	0.7282	0.8761	0.8719	0.8443	0.8716	0.7680	0.7717	0.7750	0.8531	0.8521	0.8397
	${MSSIM}_{F}^{I}$	0.2367	0.4728	0.4523	0.4756	0.4649	0.4331	0.6445	0.6941	0.6636	0.5059	0.4518	0.5964
	${MSSIM}_{F}^{V, I}$	0.5690	0.6678	0.6642	0.6737	0.6546	0.6524	0.7062	0.7329	0.7193	0.6795	0.6520	0.7180
Tank	EN	7.8740	7.4061	7.4032	7.7938	7.4472	7.9162	7.2433	7.1859	7.3162	7.5401	7.8945	7.7494
	SF	12.181	11.372	11.229	10.942	11.415	11.595	10.873	8.6654	10.037	11.348	11.706	10.394
	SD	76.021	51.331	50.908	61.049	54.508	69.035	39.379	38.575	40.701	51.008	69.487	58.459
	NMI	0.4894	0.3907	0.4265	0.5144	0.4597	0.4710	0.2272	0.4077	0.4249	0.4358	0.4007	0.4599
	${MSSIM}_{F}^{V}$	0.7539	0.6165	0.6161	0.5948	0.6129	0.6938	0.4714	0.5159	0.4600	0.6171	0.7032	0.7071
	${MSSIM}_{F}^{I}$	0.1124	0.1376	0.1446	0.2643	0.1572	0.1583	0.1915	0.2277	0.3200	0.1588	0.1776	0.1651
	${MSSIM}_{F}^{V, I}$	0.4331	0.3770	0.3804	0.4396	0.3851	0.4260	0.3315	0.3718	0.3900	0.3879	0.4404	0.4361
Keptein_1123	EN	7.2888	6.7778	6.7052	6.8575	6.7761	7.2024	6.5531	6.5170	6.5845	6.7208	7.1628	6.7485
	SF	6.5129	5.1898	5.0874	4.6267	5.3146	5.3116	4.7023	3.3338	3.9948	5.1615	6.0900	3.8423
	SD	58.915	34.158	33.615	32.898	36.264	56.012	31.619	31.454	31.477	33.880	51.728	32.069
	NMI	0.4610	0.3948	0.4134	0.4796	0.4539	0.4496	0.2776	0.4131	0.4306	0.4423	0.4000	0.4292
	${MSSIM}_{F}^{V}$	0.8274	0.6750	0.6834	0.6417	0.6626	0.6660	0.7424	0.7776	0.7318	0.6809	0.6858	0.7949
	${MSSIM}_{F}^{I}$	0.5174	0.7848	0.7948	0.8399	0.8029	0.7793	0.7681	0.7982	0.8222	0.8083	0.7548	0.7537
	${MSSIM}_{F}^{V, I}$	0.6724	0.7299	0.7391	0.7408	0.7328	0.7226	0.7552	0.7879	0.7770	0.7446	0.7203	0.7743

Method	Duine	Nato_camp
LP	$0.0043 \pm 0.0007$	$0.0043 \pm 0.0006$
LPSR	$0.0111 \pm 0.0026$	$0.0087 \pm 0.0005$
WLS	$0.0863 \pm 0.0048$	$0.0886 \pm 0.0056$
GFF	$0.0871 \pm 0.0067$	$0.0927 \pm 0.0091$
DTCWT	$0.1170 \pm 0.0093$	$0.1195 \pm 0.0022$
Proposed Method	$0.1494 \pm 0.0085$	$0.1575 \pm 0.0056$
Wavelet	$0.1550 \pm 0.0382$	$0.1592 \pm 0.0018$
MSVD	$0.1674 \pm 0.0031$	$0.1695 \pm 0.0024$
HMSD	$0.5441 \pm 0.0558$	$0.5492 \pm 0.0328$
CVT	$0.6592 \pm 0.0299$	$0.6692 \pm 0.0060$
NSCT	$1.4385 \pm 0.0092$	$1.4402 \pm 0.0096$
PC-GF	$2.7571 \pm 0.0908$	$2.7972 \pm 0.2530$

Scenes	Indexes	Proposed	CVT	DTCWT	GFF	LP	LPSR	MSVD	Wavelet	WLS	NSCT	HMSD	PC-GF
2 men in front of house	EN	6.9632	6.6638	6.6546	7.2518	6.7831	7.1301	6.4700	6.4261	6.5106	6.6949	7.0773	6.4121
	SF	7.8831	5.7364	5.7463	5.6928	5.8793	5.8303	5.3659	3.6895	5.1383	5.8504	6.4003	4.2342
	SD	53.135	21.917	27.434	44.620	30.155	52.146	23.850	23.211	24.479	27.968	45.914	21.986
	NMI	0.5860	0.6692	0.6747	0.6553	0.6794	0.6744	0.3461	0.4387	0.4801	0.4539	0.4329	0.4407
	${MSSIM}_{F}^{V}$	0.8835	0.8724	0.8789	0.9698	0.8724	0.8976	0.8294	0.8270	0.8995	0.8606	0.8718	0.7499
	${MSSIM}_{F}^{I}$	0.3726	0.5750	0.5767	0.4568	0.5891	0.5535	0.6951	0.7470	0.6048	0.6124	0.5570	0.7877
	${MSSIM}_{F}^{V, I}$	0.6281	0.7237	0.7278	0.7133	0.7308	0.7256	0.7623	0.7870	0.7521	0.7365	0.7144	0.7688
Bench	EN	7.4699	6.9685	6.9390	7.5215	7.1144	7.4337	6.6913	6.5319	6.8030	6.9607	7.3898	6.7532
	SF	12.344	10.611	10.599	10.585	10.650	10.658	10.059	7.4912	9.7313	10.600	11.034	8.2173
	SD	30.385	28.659	32.242	35.196	60.080	33.297	48.063	27.158	27.180	44.111	29.330	37.280
	NMI	0.3934	0.4604	0.5077	0.5064	0.4655	0.4594	0.5132	0.4717	0.4850	0.5905	0.5117	0.5157
	${MSSIM}_{F}^{V}$	0.7651	0.7062	0.8333	0.8445	0.8829	0.7169	0.8820	0.9119	0.9172	0.9896	0.9199	0.9106
	${MSSIM}_{F}^{I}$	0.3761	0.5205	0.3061	0.2501	0.2297	0.5121	0.3651	0.5708	0.5701	0.4742	0.5672	0.5600
	${MSSIM}_{F}^{V, I}$	0.4807	0.5387	0.5391	0.5641	0.5455	0.5459	0.5706	0.6134	0.5697	0.5473	0.5563	0.6145
Lake	EN	7.4016	6.6741	6.6578	7.3687	6.7550	7.1976	6.5683	6.5328	6.6130	6.6852	7.1569	6.5045
	SF	8.1626	5.6114	5.5899	5.7126	5.6665	5.7088	5.0750	3.1921	4.9603	5.6369	6.3033	3.5541
	SD	48.063	27.158	27.180	44.111	29.330	37.280	24.684	24.101	25.763	27.381	37.926	25.096
	NMI	0.5132	0.4717	0.4850	0.5905	0.5117	0.5157	0.3724	0.4481	0.5094	0.4802	0.4696	0.4355
	${MSSIM}_{F}^{V}$	0.8820	0.9119	0.9172	0.9896	0.9199	0.9106	0.8584	0.8410	0.9306	0.9100	0.9171	0.7904
	${MSSIM}_{F}^{I}$	0.3651	0.5708	0.5701	0.4742	0.5672	0.5600	0.7021	0.7731	0.6036	0.5889	0.5304	0.7913
	${MSSIM}_{F}^{V, I}$	0.6236	0.7414	0.7437	0.7319	0.7435	0.7353	0.7803	0.8071	0.7671	0.7494	0.7238	0.7908
Sandpath	EN	7.2040	6.4548	6.4059	6.8402	6.5898	7.1107	6.1616	6.0840	6.6190	6.4548	7.1324	6.4192
	SF	9.8059	7.0677	6.9077	6.6894	7.7551	7.7702	6.8572	4.8114	5.6727	7.4500	8.6213	5.8087
	SD	40.987	23.082	22.405	33.654	25.627	39.371	19.201	18.458	19.403	23.016	39.152	24.752
	NMI	0.5029	0.4030	0.4313	0.4982	0.4479	0.4551	0.3493	0.4085	0.4185	0.4384	0.4028	0.4380
	${MSSIM}_{F}^{V}$	0.9012	0.7282	0.8761	0.8719	0.8443	0.8716	0.7680	0.7717	0.7750	0.8531	0.8521	0.8397
	${MSSIM}_{F}^{I}$	0.2367	0.4728	0.4523	0.4756	0.4649	0.4331	0.6445	0.6941	0.6636	0.5059	0.4518	0.5964
	${MSSIM}_{F}^{V, I}$	0.5690	0.6678	0.6642	0.6737	0.6546	0.6524	0.7062	0.7329	0.7193	0.6795	0.6520	0.7180
Tank	EN	7.8740	7.4061	7.4032	7.7938	7.4472	7.9162	7.2433	7.1859	7.3162	7.5401	7.8945	7.7494
	SF	12.181	11.372	11.229	10.942	11.415	11.595	10.873	8.6654	10.037	11.348	11.706	10.394
	SD	76.021	51.331	50.908	61.049	54.508	69.035	39.379	38.575	40.701	51.008	69.487	58.459
	NMI	0.4894	0.3907	0.4265	0.5144	0.4597	0.4710	0.2272	0.4077	0.4249	0.4358	0.4007	0.4599
	${MSSIM}_{F}^{V}$	0.7539	0.6165	0.6161	0.5948	0.6129	0.6938	0.4714	0.5159	0.4600	0.6171	0.7032	0.7071
	${MSSIM}_{F}^{I}$	0.1124	0.1376	0.1446	0.2643	0.1572	0.1583	0.1915	0.2277	0.3200	0.1588	0.1776	0.1651
	${MSSIM}_{F}^{V, I}$	0.4331	0.3770	0.3804	0.4396	0.3851	0.4260	0.3315	0.3718	0.3900	0.3879	0.4404	0.4361
Keptein_1123	EN	7.2888	6.7778	6.7052	6.8575	6.7761	7.2024	6.5531	6.5170	6.5845	6.7208	7.1628	6.7485
	SF	6.5129	5.1898	5.0874	4.6267	5.3146	5.3116	4.7023	3.3338	3.9948	5.1615	6.0900	3.8423
	SD	58.915	34.158	33.615	32.898	36.264	56.012	31.619	31.454	31.477	33.880	51.728	32.069
	NMI	0.4610	0.3948	0.4134	0.4796	0.4539	0.4496	0.2776	0.4131	0.4306	0.4423	0.4000	0.4292
	${MSSIM}_{F}^{V}$	0.8274	0.6750	0.6834	0.6417	0.6626	0.6660	0.7424	0.7776	0.7318	0.6809	0.6858	0.7949
	${MSSIM}_{F}^{I}$	0.5174	0.7848	0.7948	0.8399	0.8029	0.7793	0.7681	0.7982	0.8222	0.8083	0.7548	0.7537
	${MSSIM}_{F}^{V, I}$	0.6724	0.7299	0.7391	0.7408	0.7328	0.7226	0.7552	0.7879	0.7770	0.7446	0.7203	0.7743

Method	Duine	Nato_camp
LP	$0.0043 \pm 0.0007$	$0.0043 \pm 0.0006$
LPSR	$0.0111 \pm 0.0026$	$0.0087 \pm 0.0005$
WLS	$0.0863 \pm 0.0048$	$0.0886 \pm 0.0056$
GFF	$0.0871 \pm 0.0067$	$0.0927 \pm 0.0091$
DTCWT	$0.1170 \pm 0.0093$	$0.1195 \pm 0.0022$
Proposed Method	$0.1494 \pm 0.0085$	$0.1575 \pm 0.0056$
Wavelet	$0.1550 \pm 0.0382$	$0.1592 \pm 0.0018$
MSVD	$0.1674 \pm 0.0031$	$0.1695 \pm 0.0024$
HMSD	$0.5441 \pm 0.0558$	$0.5492 \pm 0.0328$
CVT	$0.6592 \pm 0.0299$	$0.6692 \pm 0.0060$
NSCT	$1.4385 \pm 0.0092$	$1.4402 \pm 0.0096$
PC-GF	$2.7571 \pm 0.0908$	$2.7972 \pm 0.2530$

Abstract

References

Cited By

Figures (7)

Tables (2)

Equations (20)

Metrics

Login or Create Account