Hierarchical multi-granularity image classification is a challenging task that aims to tag each given image with multiple granularity labels simultaneously.Existing methods tend to overlook that different image region...Hierarchical multi-granularity image classification is a challenging task that aims to tag each given image with multiple granularity labels simultaneously.Existing methods tend to overlook that different image regions contribute differently to label prediction at different granularities,and also insufficiently consider relationships between the hierarchical multi-granularity labels.We introduce a sequence-to-sequence mechanism to overcome these two problems and propose a multi-granularity sequence generation(MGSG)approach for the hierarchical multi-granularity image classification task.Specifically,we introduce a transformer architecture to encode the image into visual representation sequences.Next,we traverse the taxonomic tree and organize the multi-granularity labels into sequences,and vectorize them and add positional information.The proposed multi-granularity sequence generation method builds a decoder that takes visual representation sequences and semantic label embedding as inputs,and outputs the predicted multi-granularity label sequence.The decoder models dependencies and correlations between multi-granularity labels through a masked multi-head self-attention mechanism,and relates visual information to the semantic label information through a crossmodality attention mechanism.In this way,the proposed method preserves the relationships between labels at different granularity levels and takes into account the influence of different image regions on labels with different granularities.Evaluations on six public benchmarks qualitatively and quantitatively demonstrate the advantages of the proposed method.Our project is available at https://github.com/liuxindazz/mgs.展开更多
Purpose: Magnetic particle imaging (MPI) allows for imaging of the spatial distribution of magnetic nanoparticles (MNPs) in positive contrast, with high sensitivity, high spatial resolution, and high imaging speed. It...Purpose: Magnetic particle imaging (MPI) allows for imaging of the spatial distribution of magnetic nanoparticles (MNPs) in positive contrast, with high sensitivity, high spatial resolution, and high imaging speed. It is necessary to increase the signal-to-noise ratio to enhance the reliability of MPI. The purpose of this study was to investigate the effect of signal filtering on the image quality and quantitativity in projection-based MPI using phantoms. Materials and Methods: We fabricated two kinds of phantom (cylindrical tube phantom with a diameter of 6 mm and A-shaped phantom) and evaluated the effect of signal filtering in terms of root-mean-square (RMS) granularity and the correlation coefficient between iron concentrations of MNPs and average MPI values for four filter modes (THRU, BPF, BEF, and LPF). In the THRU mode, the signal input was output without passing through the filter. In the BPF mode, only the third-harmonic signal was passed using a band-pass filter (central frequency: 1200 Hz, band width: 1/3 octave). In the BEF mode, the first-harmonic signal was eliminated using a band-elimination filter (central frequency: 400 Hz, band width: 1/3 octave). In the LPF mode, only the signal with a frequency less than the third-harmonic frequency was passed using a low-pass filter (cut-off frequency: 1200 Hz, -24 ± 2 dB/octave). The RMS granularity was obtained by calculating standard deviations of the pixel values in the MPI image without MNPs, whereas average MPI values were obtained by drawing a circular region of interest with a diameter of 6 mm on the MPI image of the cylindrical tube phantom. Results: When using the filtered back-projection (FBP) method with a ramp filter for image reconstruction, the RMS granularity and correlation coefficient decreased in the order of THRU, BPF, BEF, and LPF. In the BPF mode, however, some artifacts were observed. When using the maximum likelihood-expectation maximization (ML-EM) algorithm with an iteration number of 15, the correlation coefficient decreased in th展开更多
Data mining techniques are used to discover knowledge from GIS database in order to improve remote sensing image classification.Two learning granularities are proposed for inductive learning from spatial data,one is s...Data mining techniques are used to discover knowledge from GIS database in order to improve remote sensing image classification.Two learning granularities are proposed for inductive learning from spatial data,one is spatial object granularity,the other is pixel granularity.We also present an approach to combine inductive learning with conventional image classification methods,which selects class probability of Bayes classification as learning attributes.A land use classification experiment is performed in the Beijing area using SPOT multi_spectral image and GIS data.Rules about spatial distribution patterns and shape features are discovered by C5.0 inductive learning algorithm and then the image is reclassified by deductive reasoning.Comparing with the result produced only by Bayes classification,the overall accuracy increased by 11% and the accuracy of some classes,such as garden and forest,increased by about 30%.The results indicate that inductive learning can resolve spectral confusion to a great extent.Combining Bayes method with inductive learning not only improves classification accuracy greatly,but also extends the classification by subdividing some classes with the discovered knowledge.展开更多
基金supported by National Key R&D Program of China(2019YFC1521102)the National Natural Science Foundation of China(61932003)Beijing Science and Technology Plan(Z221100007722004).
文摘Hierarchical multi-granularity image classification is a challenging task that aims to tag each given image with multiple granularity labels simultaneously.Existing methods tend to overlook that different image regions contribute differently to label prediction at different granularities,and also insufficiently consider relationships between the hierarchical multi-granularity labels.We introduce a sequence-to-sequence mechanism to overcome these two problems and propose a multi-granularity sequence generation(MGSG)approach for the hierarchical multi-granularity image classification task.Specifically,we introduce a transformer architecture to encode the image into visual representation sequences.Next,we traverse the taxonomic tree and organize the multi-granularity labels into sequences,and vectorize them and add positional information.The proposed multi-granularity sequence generation method builds a decoder that takes visual representation sequences and semantic label embedding as inputs,and outputs the predicted multi-granularity label sequence.The decoder models dependencies and correlations between multi-granularity labels through a masked multi-head self-attention mechanism,and relates visual information to the semantic label information through a crossmodality attention mechanism.In this way,the proposed method preserves the relationships between labels at different granularity levels and takes into account the influence of different image regions on labels with different granularities.Evaluations on six public benchmarks qualitatively and quantitatively demonstrate the advantages of the proposed method.Our project is available at https://github.com/liuxindazz/mgs.
基金supported by the National Natural Science Foundation of China(No.60433020,No.60673099,No.60773095)the"Computing and Software Science and Technology Innovation Flat"of project 985+2 种基金the Key Laboratory for Symbol Computation and Knowl-edge Engineering of the National Education Ministry of China(No.02090)the Science and Technology Development Planning Project of Jilin Province(No.20080168)the Jilin University Graduate Innovation Foundation of"Project 985"(No.20080236)
文摘Purpose: Magnetic particle imaging (MPI) allows for imaging of the spatial distribution of magnetic nanoparticles (MNPs) in positive contrast, with high sensitivity, high spatial resolution, and high imaging speed. It is necessary to increase the signal-to-noise ratio to enhance the reliability of MPI. The purpose of this study was to investigate the effect of signal filtering on the image quality and quantitativity in projection-based MPI using phantoms. Materials and Methods: We fabricated two kinds of phantom (cylindrical tube phantom with a diameter of 6 mm and A-shaped phantom) and evaluated the effect of signal filtering in terms of root-mean-square (RMS) granularity and the correlation coefficient between iron concentrations of MNPs and average MPI values for four filter modes (THRU, BPF, BEF, and LPF). In the THRU mode, the signal input was output without passing through the filter. In the BPF mode, only the third-harmonic signal was passed using a band-pass filter (central frequency: 1200 Hz, band width: 1/3 octave). In the BEF mode, the first-harmonic signal was eliminated using a band-elimination filter (central frequency: 400 Hz, band width: 1/3 octave). In the LPF mode, only the signal with a frequency less than the third-harmonic frequency was passed using a low-pass filter (cut-off frequency: 1200 Hz, -24 ± 2 dB/octave). The RMS granularity was obtained by calculating standard deviations of the pixel values in the MPI image without MNPs, whereas average MPI values were obtained by drawing a circular region of interest with a diameter of 6 mm on the MPI image of the cylindrical tube phantom. Results: When using the filtered back-projection (FBP) method with a ramp filter for image reconstruction, the RMS granularity and correlation coefficient decreased in the order of THRU, BPF, BEF, and LPF. In the BPF mode, however, some artifacts were observed. When using the maximum likelihood-expectation maximization (ML-EM) algorithm with an iteration number of 15, the correlation coefficient decreased in th
文摘Data mining techniques are used to discover knowledge from GIS database in order to improve remote sensing image classification.Two learning granularities are proposed for inductive learning from spatial data,one is spatial object granularity,the other is pixel granularity.We also present an approach to combine inductive learning with conventional image classification methods,which selects class probability of Bayes classification as learning attributes.A land use classification experiment is performed in the Beijing area using SPOT multi_spectral image and GIS data.Rules about spatial distribution patterns and shape features are discovered by C5.0 inductive learning algorithm and then the image is reclassified by deductive reasoning.Comparing with the result produced only by Bayes classification,the overall accuracy increased by 11% and the accuracy of some classes,such as garden and forest,increased by about 30%.The results indicate that inductive learning can resolve spectral confusion to a great extent.Combining Bayes method with inductive learning not only improves classification accuracy greatly,but also extends the classification by subdividing some classes with the discovered knowledge.
