A fullautomatic cloud detection algorithm based on deep semantic segmentation is proposed to improve the accuracy of cloud detection for the remote sensing imagery of FY2E satellites. Firstly, to train and evaluate, a sample data set is created by the data of FY2E L1 matched with the cloud detection results with high accuracy. Secondly, a deep semantic segmentation network is designed. A loss function is improved to extract the cloud’s boundary effectively for a severe imbalance between positive and negative samples in the train data set. Finally, FY2E and MODIS data, taken as train and label samples, respectively, are used for training networks, resulting in four classification models for detecting FY2E L1 imagery. The test results show that the proposed method’s accuracy and the Kappa coefficient are 75% and about 0.53 in four classification tests, respectively. Compared with the existing multichannel threshold method in two classification tests, the proposed method can improve the accuracy of about 90% of the samples and the accuracy of some samples by more than 20%. In addition, the proposed method has a strong recognition ability for cloud edges, broken clouds and other details. It has a certain degree of robustness, which is less affected by the misclassification categories in train samples. Furthermore, by expanding the data set and optimizing the network, the proposed method will improve the data quality of the entire disk imagery of FY2.