Deep learning is showing a growing trend in data analysis and is known as one of the 10 breakthrough technologies in 2013 [1]. It is an improvement to the neural network, including more computational layers, enabling higher levels of abstraction and prediction in the data [2]. So far, it is becoming a leading machine learning tool in the field of general imaging and computer vision.
In particular, Convolutional Neural Networks (CNN) have proven to be an advantageous tool for many computer vision tasks. Deep CNN can automatically learn intermediate and advanced abstract concepts derived from raw data (eg, images). Recent results show that the generic descriptor extracted from CNN is very effective in object recognition and localization of natural images. Medical image analysis groups around the world are rapidly entering the field and apply CNN and other deep learning methods to a wide range of applications. Many good results are emerging.
In the field of medical imaging, the exact diagnosis or assessment of a disease depends on image acquisition and image interpretation. In recent years, with the development of technology, devices can collect data at a faster rate and with a more powerful resolution, which greatly improves the quality of image acquisition. However, the improvement of image interpretation by computer technology has only just begun. At present, most medical image interpretations are performed by doctors. However, human interpretation of images is often one-sided because of its subjectivity, large changes in the different interpreters, and fatigue. Many diagnostic tasks require an initial search process to detect abnormalities and quantify changes in measurements and time. Computerized tools, especially image analysis and machine learning, play a key role in improving diagnosis. They support expert workflows by helping identify areas that need treatment. Among these tools, deep learning has quickly confirmed its superiority as a basis and can improve accuracy. It also opens up new areas for data analysis and continues to develop at an unprecedented rate.
A. Historical Network
The basic idea behind neural networks and deep learning has existed for decades [3]. They are usually only a few layers. The appearance of the back-propagation algorithm has significantly improved the performance of neural networks. However, performance is still not enough. Other classifiers have been gradually developed, including decision trees, boosTIng, and support vector machines. Each of them has been applied to medical image analysis, especially for detecting abnormalities, and they have also been applied in other related fields such as segmentation (segmentaTIon). Despite this development, relatively high false positive rates are still common.
As early as 1996 in the work of Sahiner et al., CNN (convolutional neural network) was applied to medical image processing [4]. In this work, ROIs (Region of Interests) containing biopsy-proven masses or normal tissues were extracted from mammograms. The CNN contains one input layer, two hidden layers, and one output layer and the back propagation used. In this pre-GPU era, training time was described as "computation-intensive" but no specific time was given. In 1993, CNN was used for lung nodule detection [5]. In 1995, CNN was used to detect microcalcification on mammograms [6].
A typical CNN for image processing has a structure that consists of a series of convolution filter layers interspersed with a series of data compression or pooling layers. A convoluTIon filter processes a small block of the input image. Similar to the low-level pixel processing of the human brain, the convolution filter can detect highly correlated image features, such as lines or circles that can represent sharp edges (for example, for organ detection) or circles (such as objects for circles). Like colon polyps, then high-level features such as local or global shapes and textures. The output of the CNN is usually a label of one or more probabilities or classes corresponding to the images. The convolution filter can learn directly from the training data. This is exactly what people need because it reduces the need for very time-consuming manual marking features. Without the use of convolution filters, filters designed for specific applications and some features that need to be calculated are inseparable from these artificial features during the preprocessing of the image.
CNN is a highly parallelized algorithm. Compared to individual CPU processing, a large part of the practicality of using CNN comes from the huge speed increase (approximately 40 times) contributed by the image processing unit (GPU). Early papers describing the value of GPUs for training CNN and other machine learning technologies were published in 2006 [8]. In medical image processing, GPUs were first introduced for segmentation, reconstruction, and registration, followed by machine learning [9], [10]. Interestingly, although Eklund et al. [10] widely discussed convolutions in their 2013 paper, convolutional neural networks and deep learning were not mentioned at all. This highlights how rapidly the major reforms in deep learning have rapidly adjusted medical image processing research.
B. Today's Network
Due to the development of new variants of CNN and the emergence of parallel solvers for modern GPU optimization, deep neural networks have recently gained considerable commercial interest. The power of CNN is due to its deep architecture, which allows it to extract a set of distinguishing features at different levels of abstraction. Training a deep convolutional neural network from scratch is a huge challenge. First, CNN requires a large amount of tagged data, which is difficult to achieve in the medical field. This is because it is very expensive to ask experts to mark, and samples of diseases (such as lesions) are scarce. Second, training depth CNN requires a lot of computational and memory resources. Without them, the training process can be very time consuming. Third, training a deep CNN is often complicated by overfitting and convergence problems, and it is often necessary to repeatedly adjust the learning parameters or architecture of the network to ensure that all layers learn at a considerable speed. In view of the above difficulties, some new learning schemes called "transfer learning" and "fine-tuning" have been proposed to provide solutions and are being accepted by more and more people. These will be discussed further in Section II-C.
C. Network in the medical field
The domain deep learning method is most effective when applied to large training sets, but in the medical field, large data sets are not always available. Therefore, we face a series of major challenges, including: (a) Can deep neural networks be effectively used in medical tasks? (b) Is transfer learning from general imagery to the medical field relevant? (c) Can we rely solely on the characteristics of learning, or can we combine them with artificially produced functions to accomplish tasks? This special issue of IEEE imaging (IEEE-TMI) for deep learning of medical imaging focuses on the advancement of this new era of machine learning and its role in the field of medical image processing. This question describes the recent achievements of CNN and other deep learning applications in medical tasks. It contains 18 articles selected from 50 papers of various investigators from all over the world. This is a very high number for IEEE's special problems, and this is the ratio of the time from the publication solicitation to the submission deadline. It was achieved within a short time in the past. The paper focuses on a large number of traditional tasks from detection to categorization (eg, lesion detection, image segmentation, shape modeling, image registration), as well as some open and novel application areas. It also includes some work focused on network exploration and gives an idea of ​​how different tasks, parameters, and training sets should be selected for the architecture.
Overview of journal articles and topics
A. Lesion detection
Computer-aided detection (CAD/Computer-aided detecTIon) is a complete medical image analysis field that is well suited for deep learning. In the standard method of CAD [11], lesions are detected by supervised methods or classical image processing techniques such as filtering and mathematical morphology. Candidate lesions are often segmented and are often described by a large number of manually designed features. The classifier maps the feature vector to the probability that the corresponding candidate site is an actual lesion. A straightforward approach to using deep learning instead of manually designed features is to train CNNs that operate on a set of image-image data centered on candidate lesions. Several articles in this issue have used this method. To obtain candidates for pulmonary nodules from 3D chest CT scans and to extract 2D patches with 9 different orientations centered on these candidate locations, Setio et al. [12] combined three previously developed probe candidates. Combine different CNNs to classify each candidate. The report shows that this method has achieved a slight improvement compared with the results of the previously published classic CAD for the same task.
Roth et al. [13] used CNN to improve three existing CAD systems for detecting the presence of colon polyps in colonoscopic CT and the use of volume CT to detect hardened spinal metastases and enlarged lymph nodes. They also used previously developed candidate detectors and 3 orthogonal 2D patches, as well as up to 100 random rotation views. The randomly rotated "2.5D" view is a way of decomposing the representation of the image from the original 3D data. Additional accuracy gains are then obtained by integrating CNN's predictions on the 2.5D view. For all 3 CAD systems using CNN, the sensitivity of lesion detection improved by 13 – 34%, indicating that the method is universally adjustable. With non-deep learning classifiers (such as support vector machine families), it is almost impossible to achieve this level of improvement.
Dou et al. [14] detected brain microbleeds from magnetic susceptibility-weighted magnetic resonance imaging scans. Using a 3D CNN and replacing the original candidate detection phase with CNN, they proposed a two-phase approach. The report proposes to reimplement, train and test on the same data set. The results of their 3D CNN have been improved compared to other classical methods and 2D CNN methods in the existing literature.
Sirinukunwattana et al. [15] detected and classified nuclei in histopathological images. They use a CNN that uses a small block as input, rather than just predicting if the center pixel of the patch is the nucleus. They model the output, and each nucleus center produces a peak that is flatter elsewhere. This combination of spatially constrained CNNs and overlapping patches in the testing phase yields better results than previous previously proposed techniques based on CNN and classical signature methods.
Anthimopoulos et al. [16] focused on the use of 2D chest CT scans to detect patterns of interstitial lung disease. They are one of the three groups that studied the problem (the other two were Shin et al. [17] and van Tulder et al. [18]) using the public dataset from [19]. They trained CNN so that they could distinguish which of the 7 classes the 32 × 32 pixel tile belongs to. The report shows that their results achieved higher accuracy than the previous three methods that used manual design features.
Lesions detection is also an interesting topic in several other articles dealing with such issues, but the focus of these articles is on a broader or focused approach to specific methodological issues. These papers will be briefly discussed below.
B. Segmentation and Shape Modeling
For a large data set consisting of 2,891 echocardiograms, Ghesu et al. combined deep learning with edge-space learning for object detection and segmentation. The combination of “effective exploration of large-parameter spaces†and a method of enhancing sparsity in deep networks has increased computational efficiency, and the method has reduced the average segmentation error by 13.5% compared to another reference method published by the same group. .
There are three groups of researchers who focus on brain structure or brain lesions. The magnetic resonance imaging (MRI) problem of segmentation of multiple sclerosis brain lesions was solved by Brosch et al. They have developed a 3D deep convolutional coding network that can incorporate interrelated convolution and deconvolution processes. The convolution process learns higher levels of functionality while the deconvolution process predicts voxel level splitting. They applied this network to two common datasets and one clinical trial dataset and compared their own methods with five public methods. The report said that the performance of the method "can be comparable to the current state-of-the-art methods."
Pereira et al. studied brain tumor segmentation in magnetic resonance imaging. [ twenty two ]. They use a small kernel, a deeper architecture, grayscale normalization, and data enhancements. Different convolutional neural network architectures are used for low-grade and high-grade tumors. This method separates the enhanced and the core parts of the tumor. They ranked first in the 2013 Dataset Public Challenge and ranked second in the 2015 Live Challenge.
For the problem of brain structure segmentation, a study by Moeskops et al. showed that the convolutional neural network performed well on datasets covering five different age groups of patients from preterm to old age. A multi-scale method is used to achieve its reliability. This method has achieved good results in eight organizational categories, of which the average Dice similarity coefficient for five datasets is 0.82 to 0.91.
C. WebQuest
1) Data Dimensional Problem - 2D vs 3D: Most of the data studies we have seen use 2D analysis. The two-dimensional to three-dimensional transition is often questioned - whether it will be a key to a significant increase in performance. There are some changes in the data enhancement process, including 2.5 dimensions. For example, in Roth et al.'s study, axial, coronal, and sagittal images centered on candidate colon polyps or voxels in lymph nodes and entered into a cuda-convnet convolutional neural network. The network contains Usually used to represent a natural light image of the red, green and blue channels. Three-dimensional convolutional neural networks were explicitly used by Brosch and Dou et al.
2) Learning Methodology - Unsupervised vs. Supervised: When we look at online literature, it is clear that most studies focus on supervised convolutional neural networks in order to achieve classification. This kind of network is very important for many applications, including detection, segmentation, and marking. However, there are still some studies that focus on unsupervised solutions. Most of them have proved to be useful in image coding, efficient image representation planning, and as a preprocessing step for in-depth monitoring. Unsupervised characterization learning methods such as Restricted Boltzmann Machines (RBM) may exceed the standard filter banks because they learn characterization directly from the training data. RBM is trained by a generative learning target; this allows the network to learn representations from untagged data, but it does not necessarily produce the best classification features. Van Tulder et al. conducted a survey that combines the advantages of generating and discriminating learning objectives in volume-integrated RBM. This study shows that the combination of learning tasks is superior to pure discriminative or generative learning.
3) Training data Note: The convolutional neural network realizes the learning of data-driven, highly-representative, hierarchical hierarchical image features. In many application areas (see the journal), these features have proved to be a very powerful and reliable characterization. To provide such a rich characterization and successful classification requires enough training data. The amount of data required is a key issue to be explored. Related issues include the following: How can we use the training data we have most effectively? What can we do without data? And finally, is there an alternative to obtaining data and making medical annotations?
Some of these issues were resolved by some of the papers in the journal. Van Grinsven et al. attempted to improve and accelerate convolutional neural network training for solving medical image analysis tasks by dynamically selecting negatively classified negative samples during training. The convolutional neural network training process is a continuous process that requires multiple iterations (or multiple periods) to optimize the network parameters. At each stage, a subset of samples is randomly selected from the training data and presented to the network to update its parameters by backpropagating and minimizing the cost function. The classification task in the medical field is often a normal/pathological type of discrimination task. In this case, the normal type category is particularly over-characterized; furthermore, due to the repetitive pattern of normal tissue in each image, most normal type training samples are highly correlated. Only a small part of it contains useful information. Equivalent processing of these data during the learning process can result in a waste of many training iterations on useless normal samples, making the convolutional neural network training process take unnecessary time. One method that can identify useful normal samples, as shown in this study, improves the efficiency of the convolutional neural network learning process and reduces training time.
4) Migration learning and fine-tuning: Obtaining data in the medical imaging field as fully annotated as ImageNet remains a challenge. When there is not enough data available, there are several ways to help: 1) Migration learning: A natural image data set or a convolutional neural network model (supervised) pre-trained in a different medical field is Used for a new medical task at hand. One solution is that a pre-trained convolutional neural network is applied to an input image, and its output is then extracted from the network layer. The extracted output is taken as a feature and used to train a separate pattern classifier. For example, in Bar et al.'s study, a pre-trained convolutional neural network was used as a feature generator to identify chest pathology. In the Ginneken et al. study, features based on convolutional neural networks and manually added features were integrated to achieve a performance improvement in a nodule detection system. 2) Fine-tuning: There is indeed a medium-sized data set available for the task at hand, and a reference scheme is to use a pre-trained convolutional neural network as several (or all) network layers after further supervised training is completed. Initialize to use, use new data in the task at hand.
Migration learning and fine-tuning are key parts of using deep convolutional neural networks in medical imaging applications. It is the research work of Shin and Tajbakhsh that discusses these issues. The experimental results in the study consistently show that the use of pre-trained neural networks with fine-tuning can achieve the best results, both for specific application areas (Tajbakhsh et al.) and for all network architectures (Shin et al.). Further analysis by Tajbakhsh et al. shows that depth fine-tuning is superior to shallow fine-tuning in terms of performance improvement, and the reduced size of the training set also increases the importance of using fine-tuning. In the study of Shin et al., the Google Net framework achieved the most advanced mediastinal lymph node detection compared to other shallower deep architectures.
5) Ground Truth - derived from experts and non-experts: The lack of publicly-authenticated data, and the difficulty of collecting this data in every medical mission, plus cost and time overhead, these are all medical The prohibitive limiting factor. Although crowdsourcing implements annotations for large databases of real-world images, its application for biomedical purposes requires a deeper understanding and requires a more precise definition of the actual annotation tasks (Nguyen and McKenna et al. ). The fact that expert tasks are outsourced to non-expert users may lead to cluttered comments, causing disagreements among users. Many problems arise in the combination of knowledge of medical experts and non-professionals, such as how to combine information sources, how to evaluate and mix input weights by their previously proven accuracy in performance and other aspects. These problems were solved by Albarqouni et al. They propose a network that incorporates an aggregation layer, which is integrated into a convolutional neural network, thus making learning input originating from the crowd annotated as part of the network learning process. The display results provide valuable insights into the functions of deep convolutional neural network learning. The most striking fact about crowdsourcing research in the medical field is actually the conclusion that a group of non-professional, inexperienced users can actually do as well as medical experts. Nguyen and McKenna et al. also observed this in the study of radiological images.
D. Innovative Applications and Novelty Cases
Kallenberg's work [32] is based on mammographic X-ray images as an input source, using unsupervised feature learning to score mammary gland disease risk. They showed a way to learn hierarchical features from unlabeled data, and these features would then be entered directly into a simple classifier. In this classifier, two different types of operations will be performed: 1) image segmentation of the breast density, and 2) scoring of mammography X-chip textures. Classifiers perform very well in both areas. In order to control the capacity of the training model, sparse time and scope are controlled through a sparse regularization optimization. In the unsupervised learning process, the convolution layer can actually be viewed as an autoencoder. In the supervised learning part, the (pre-trained) weights and bias values ​​are further fine-tuned using the softmax regression function.
A work of Yan[33] et al. designed a multi-stage deep learning framework to deal with image classification problems and apply it to local human feature recognition. In the pre-training stage, the convolutional neural network is trained by multi-instance learning so as to obtain the local tiles with the most discriminative local tiles and invalid messages in the current training data slice. In the intensive phase, the pre-trained convolutional neural network will further train the image classifier through corresponding partial images, thereby enhancing his classification ability. The highlight of this multi-sample deep learning method is the automatic completion of the identification of local images of differentiating partial images and invalid messages. Therefore, no manual labeling work is required in advance.
The use of regression networks in medical images is not very common. Miao et al. proposed a regression network based on a convolutional neural network to achieve real-time 2D/3D registration. They proposed three algorithms to simplify the potential mapping object regression and added a strong nonlinear model to the CNN regression model. From the output of this network, the deep learning algorithm is more accurate and robust than the previous optimal algorithm, and greatly improves the process of 2D/3D registration based on grayscale.
Currently we are still exploring the areas where neural networks can be applied, and in which areas their applications and mission dimensions will have a lasting impact. In a groundbreaking study, Golkov [35] proposed a primordial argument. He used deep learning to simplify diffusion MRI (images of magnetic resonance) image processing and optimize it only after one step. Their research shows that this improvement allows people to obtain scalar measurement data from a sophisticated model with a 12-fold reduction in scan time, and it can identify abnormalities without the use of a diffusion model. Revealing the relationship between diffusion-weighted signals and microstructure characteristics is worth looking into. Golkov [35] stated that the use of deep neural networks may reveal such a relationship: DWIs can be directly used as input data instead of scalar measurements obtained by model fitting. This study shows that microstructure prediction based on voxel-by-cube pixels and automatic model-free image segmentation based on diffusion-weighted imaging values ​​can be used for model training of healthy tissue and MS lesions. Diffusion peaks are widely known by the density estimates of 12 data points, diffused bulges, and only 8 data points. This provides a quick and robust method for clinical research, and also shows that standard data processing can be simplified with deep learning.
Discussion: Key issues and outlookMany existing work shows that the use of in-depth networks has increased the current highest level, and these improvements are consistent in many areas. Under normal circumstances, the progress made by deep learning to provide solutions is relatively straightforward. We can see this obvious progress in the field of medical computing. In the article “In-depth Learning in Medical Imaging: An Exciting New Technology Review and Outlookâ€, some questions were raised: The 2012 large-scale species recognition theory had a 10% improvement, but how was it applied? Have you achieved a substantial leap forward? Is the question raised correctly? Is the direction of exploration correct? Is the imagery used to support it (eg 2D or 3D)? Need to get more data from each medical case? Or is it turning to deep learning more efficient? There are more related issues that were raised in the second part of this article. Most of the problems need to be solved.
In this paper, it can be seen that although supervised learning and unsupervised learning are accessible through deep networks, it seems that most of the work is using supervised learning. What about medicine? The amount of data is a key factor in the form of combining the advantages of both supervised learning and unsupervised learning. In the medical field, because it is difficult to obtain big data (manual annotation is difficult to obtain), the field needs more semi-supervised learning and supervised learning.
This article includes many network architectures. It can be seen from the current published paper that variability is very large. Choosing a known architecture, designing a stable framework for tasks, cross-architecture integration, etc., can all lead to variability. We can ask an interesting question in this regard: If a very deep residual network crosses the 152th layer and performs best at the ILSVRC 2015 level classification task, can it be applied to medical care to obtain good results?
A very important aspect of deep learning is that it can benefit from a large amount of training data. After the ILSVRC contest based on the ImageNET data set, a great breakthrough in computer vision was achieved (). Compared to the training datasets and test datasets used in other papers, the datasets used for this particular problem are very large (million and 1100). If we can build such a large collection of public medical image data, our society will benefit greatly.
Why is this job very challenging? First, it is difficult to raise funds for the construction of such a data set; secondly, high-quality annotation of medical image data requires medical expertise, which is not only very scarce and very expensive; third, compared with natural images, Privacy issues make medical data more difficult to obtain; fourth, the breadth of medical imaging needs to gather more different data sets. Despite all these obstacles, we have made rapid progress in data collection and data sharing. Many public data sets have been released and are now used in practice. For example, VISCERAL and the Cancer Imaging Archives, Roth et al. [13] and Shin et al. [17] have obtained datasets from analysis of enlarged lymph node images from CT scans that have been published in cancer imaging archives. The group also disclosed pancreas data sets on the line.
Since 2007, it has become a habit to hold competition seminars at medical imaging conferences such as MICCAI, ISBI, and SPIE Medical Imaging. There are a large number of datasets and ongoing research () on the website. Using these common benchmark datasets has clear advantages over using only public datasets: the race provides a precise definition of the task to be solved, and one or more assessment metrics have been defined to provide fair evaluation criteria for various algorithms. Without such evaluation criteria, it is difficult to compare different methods of the same problem even if each algorithm uses the same data set. For example, three of these studies (Anthimopoulos et al. [16], Shin et al. [17] and van Tulder et al. [18]) used the same set of CT scan data sets for chest interstitial lung disease with medical annotations, but they reported the results. It's different.
In this regard, a study on this issue (Setio et al. [12]) has seen initial success in a challenge to pulmonary nodules. This challenge was co-organized by the IEEE and ISBI conferences using an open LIDC/IDRI data set. The system proposed in this article can be compared directly with its alternative methods.
Last year, there was a game based on machine image learning based on medical image analysis. Kaggle organized a competition to identify color images of the eyes of diabetic patients with a bonus of $100,000. 661 teams submitted results and provided a total of 8,000 images. This data was used for a special study (van Grinsven et al. [24]). recent. The second session measured the volume of the heart through MRI images, and the ejection fraction image medical image analysis contest was over. A total of 192 teams took part in the competition with a bonus of $200,000. In both competitions, the best competitors are using convolutional neural networks. Among the better algorithms used, contestants using large data sets and deep learning showed greater advantages. We hope this trend can continue. In this case, the following series of worldwide competition to improve the accuracy of various imaging cancer screenings may arouse the attention of relevant people.
Research by Albarqouni et al. shows that online platforms, such as those in games, can be used for a variety of purposes. They will promote new cooperation, form solutions, and be able to obtain large amounts of data through crowdsourcing.
Bluetooth louder speaker has a high-power trumpet, the subwoofer is very shocking, and it can also realize TWS wireless serial connection, the left and right channels can play songs at the same time, as if you are in a theater.
The portable Bluetooth rich bass speaker is very smart. It can support AUX, USB, TF card memory reading, Bluetooth connection, etc. for playback. The LED light flashes according to the rhythm of the music, which is very sensational and cool!It supports hand free call then frees your hands.
Bluetooth Speaker also has a long battery life. The 6000mAh battery allows the playback time to reach 5H~10H, allowing you to enjoy music anytime and anywhere when you outdoors without worrying about the battery running out quickly.
We also use waterproof fabric to make it, the waterproof Bluetooth speaker grade reaches IPX5, which can prevent rain or domestic water from splashing on the speaker. When using it, remember to close the charging rubber plug,Dont worry about ranning or splashing water,let you enjoy the music.
Waterproof IPX5 Bluetooth Louder Speaker,Portable Bluetooth Louder Speaker,Heavy Bass Bluetooth Louder Speaker
Shenzhen Focras Technology Co.,Ltd , https://www.focrass.com