Spectral-spatial analysis of remote monitoring data to identify agrophytocenoses stresses of technological natureN.A. Pasichnyk*, O.O. Opryshko and D.S. Komarchuk
Methods of spectral-spatial analysis are promising for the identification of technological stresses. The most common solution for interpreting the causes of stress is the use of machine learning technologies, namely neural networks. As at technological stresses in particular at chemical poisoning of crops, there can be various options of the coloring of the affected plants the possibility of providing a sufficient amount of initial data for training of neural networks is doubtful. An alternative is graph analysis of the distribution of stress areas on the field map. Given the urgency of the problem for promising technologies of precision agriculture, the work aimed to develop a spectral-spatial method of monitoring technological stresses, namely the algorithm and software for its.
Experimental studies of the manifestation of technological stresses on winter crops on the example of wheat and rapeseed were conducted during 2018-2020 in production fields using universal cameras in the visible range and special multispectral Slantrange systems.
For remote monitor the state of winter crops an algorithm for identifying technological stresses was developed, which is implemented in the developed software in Python for spectral-spatial analysis of stress index maps. It has been experimentally confirmed in the production fields that the use of the developed software allows identifying the contours of areas of plants with stresses of technological nature based on stress index distribution maps.
Agrophytocenoses stresses, UAVs, Winter crops, Vegetation indices.
In recent years, there has been a steady trend towards increasing the cost of food in general, and especially cereals and oilseeds. Today's food supply is facing environmental issues, climate change and rising resource costs. All this became a prerequisite for the concept of precision agricultural production, which provides for the transition of agricultural production to a new technological level through, first of all, the introduction of automation technologies at all stages of crop production. Biological objects are extremely difficult to automate due to multifactoriality the influence of weather conditions, soil heterogeneity, technological methods, etc. This necessitates a methodological justification of remote monitoring systems as an integral part of precision agricultural production.
Traditional methods of ground research have been developed methodically, but have limitations in efficiency, scaling, availability, cost. Even for greenhouses, where the factors of crop formation are maximally artificially regulated, the use of sensors of technical vision is proposed, as shown in the works of V. Lysenko et al., (2020), I. Korobiichuk et al., (2017). For field phytocenoses, the use of stationary visual sensors is impractical, justified solutions are satellites and UAVs that can provide scalability, but they have some problems with data reproducibility, and most importantly-the interpretation of the results. Remote monitoring technologies, both at the state level and for large agricultural enterprises, have been developed since the Earth Resources Observation Satellites program (renamed Landsat in 1975) established in the United States in 1966. Positive results have been achieved in the identification of objects in satellite images. In the work of O. H. Tarariko et al., (2017) presents calculations, based on satellite monitoring data, long-term forecast of the impact of climate change on grain yields.
The prospects for agricultural production management based on objective remote monitoring data were obvious both at the state level and for agricultural enterprises. Accordingly, research was carried out to develop various theories and methods for obtaining information about vegetation. Under uncertainty, M. Lotfi et al., (2009) proposed computer data processing systems for satellite data filtering and machine learning technology for object recognition. That is, in the spectral-spatial analysis, the field of the field as a whole was not considered as the object of research. This approach is used in particular in aviation for the implementation of orientation in the use of electronic warfare as shown in the work of S. Shvorov and others (2018). Regarding agricultural production, Xianlong Zhang and others (2019) proposed the division of spectral-spatial monitoring methods into 2 conditional categories. The first category uses the spectral characteristics of terrestrial objects and then obtains vegetation information by comparing the difference with the results of spectral monitoring. An example of such monitoring is the identification of trees in densely populated cities based on satellite images shown in S.W. Myint et al., (2013).
The second category is based on a combination of external knowledge such as decision tree for image classification shown in Andrea S. Laliberte et al., (2007), neural networks, and wavelet transforms described, Bryson M. et al., (2010). This promising method has not been widely used in satellite monitoring because the combination of time delay and low resolution leads to unacceptably large errors, which was shown in the article by Dorji P. et al., (2017). UAV monitoring is devoid of these shortcomings and accordingly, this method can be implemented on a new technological basis. Thus, in the work of J. Senthilnath et al., (2017), it was possible to successfully identify weeds in crops by fixing plants in automatically determining technological tracks.
Wavelet analysis methods do not require the division of the image into blocks, because the required localization properties are already embedded in the wavelet system. Accordingly, it is possible to filter out a significant number of errors inherent in pixel analysis methods Pasichnyk, N. (2021). The proposed solution proved its effectiveness when, as a result of a dosing error in a part of the field, a higher dose of herbicides was applied. Since the application was made by appropriate ground equipment, the authors in the analysis of the map image focused on the search for linear functions. The authors noted some difficulties in the established systems when choosing thresholds. The complexity of this controversial issue was confirmed by Yu-Hsuan Tu et al., (2020) where it was the limit values that were recommended to be studied at higher resolutions. In our opinion, this approach also has limitations. Significant scale of impressions can be easily identified by satellite technology or ground-based monitoring, but small areas will be difficult to identify, and this is important for production. Crop management technologies need to be adapted to respond to such problem areas, as weakened plants are easily affected by pests and can become a breeding ground for them.
A possible technology for the analysis of spectral-spatial distribution is artificial neural networks which, due to the rapid development of multi-core processors, have become available to farmers. There is a positive experience of using neural networks for various monitoring tasks which, if necessary, can be adapted to monitor technological stresses. Section 1 shows that in the initial stages of the growing season, the dimensions of plants may indicate their stress. Neural networks for estimating plant height during rice lodging are shown in Ming-Der Yang et al., (2020). According to the provided results, it was possible to detect rice lodging with acceptable accuracy based on images from universal cameras in the visible range, but the calculations were performed using cloud services, which is difficult to implement in our country. Autonomous work of neural networks is shown to identify grass among general vegetation W. Gruszczyński (2019). When analyzing the image was segmented into parts and carried out training of the network on the distribution of the cloud of points. This approach is promising for the identification of low-growing grass because only one manifestation is considered, but under technological stress, there may be more. In principle, for neural networks, there can be several options for identifying objects. A fundamental limitation for the training of neural networks of object identification can be several options, as shown in the example of the identification of apples in the work of I. Smirnov et al., (2021). They can be used in particular to determine the state of rice yield at the stage of ripening Qi Yang, (2019), or the state of mineral nutrition described Lysenko, V., (2017). Spatial distribution was also considered to calculate the number of plants in a ridge Y. Pang, (2020). All these works are combined by a limited number of classification options and a large sample of source data for neural network training. In this case, in contrast to the vegetation indices, which focus on pixel-by-pixel analysis, the training of neural networks was based on crop areas obtaining more accurate results.
Since technological stresses, in particular due to the phytotoxic action of chemicals, can contain various color options for affected plants, the possibility of providing a sufficient amount of initial data for learning neural networks is questionable.
There are no ready-made software solutions for analyzing of the distribution of stress areas on the field map to identify the nature of stress. Given the urgency of the problem for promising technologies of precision agriculture, the aim of the work was to develop a spectral-spatial method of monitoring technological stresses, namely the algorithm and software for its implementation.
The state of the issue
Identification of direction of movement of technological equipment: Stressful conditions of crops of technological character are caused by human actions which are realized by the use of the ground technological equipment. The identification of equipment directions was considered in Junfeng Gao et al., (2018), regarding the detection of weeds in row crops, where all plants between rows were considered weeds. In Carlos Henrique Wachholz de Souza et al., (2017), sugar cane rows were identified to estimate row gaps. In both cases, the rows were considered to be the arrangement of plants in a row, because this is how ground equipment moves. However, in agricultural practices, the directions of ground equipment movement should change from year to year, and, accordingly, the distribution of stress areas may differ from the direction of crop rows. Accordingly, the identification of stresses can be based on the assessment of the contour of the stress section, which for technological stresses must have the correct geometric shape inherent exclusively in artificial objects. In particular, in the case of chemical poisoning of plants, the boundary between affected and healthy crops will be directly linear.
Choice of the software environment: Assessing the nature of stress for crops is an urgent task to be solved both by agronomists directly in the fields and by relevant specialists using cloud services. Accordingly, for the versatility of the operating system used, it is advisable to use a cross-platform programming language such as Python, which is adapted to the fate of large data processing and machine learning. In the work of Emad Ebeid et al. (2018), devoted to the review of flight controllers and flight control of UAVs, the prospects of the Python language for these tasks were emphasized primarily due to the use of technical means from different manufacturers on different operating systems.
Experimental research: Experimental studies of the manifestation of technological stresses on winter crops on the example of wheat and rapeseed were conducted during 2018-2020 in production fields. Photography was performed using: in 2018-2019, to monitor the stationary experiment and fragments of production fields-hexacopter based on multi-rotor platform CD 600 with a set of specialized sensor equipment in the digital action camera GoPro HERO4, in 2019-2020-multispectral system Slantrange 3p, mounted on a DJI Matrice 600 hexacopter, which allowed to obtain orthophotos of industrial fields. It is the spectral-spatial analysis of the obtained orthophoto plan that allowed us to establish the dependences on the basis of which the identification of technological stresses is carried out.
Technological stresses on winter oilseed rape can be detected by means of leaf diagnostics because in September-November there is an abnormal color of the lower leaves, which is easy to establish both by ground visual assessment and research using UAVs. For winter wheat, such manifestations suitable for reliable identification from the UAV platform on an industrial scale (height from 60 meters) could not be detected. In ground-based monitoring, it was noted that plants have a characteristic deformation of the leaves, which can be a characteristic criterion for identifying the nature of stress. Affected areas inside the field are more dangerous for industrial fields, which are difficult, often impossible to visually detect by ground monitoring means. This situation is extremely dangerous, as areas with weakened plants appear in the field, which is more susceptible to pests and can become centers for the spread of the latter. Accordingly, it is advisable to develop a technology that will identify stress areas regardless of their location. This was taken into account when choosing the experimental production field.
Analytical research: Laboratory studies accompanied all stages of plantation monitoring. A sampling of plants and soil was performed on the day of monitoring or within two days thereafter. Soil samples were taken from a layer of 0-25 cm, prepared for analysis according to DSTU ISO 11464: 2007. Agrochemical analysis was performed in scientific and research laboratories of the Department of Agrochemistry and Plant Product Quality, Ukrainian Laboratory of Agricultural Products Quality, in compliance with accepted methods and techniques.
Algorithm for identification of technological stresses, its software implementation, and results of experimental data processing
Select the source data format: To process spectral monitoring data, the Slantrange sensor system has its own Slantview software, which allows you to save the received maps in several data formats, namely Shapefile, KMZ, GeoTiff. The shape format contains attributive information of geometric objects and is designed primarily to create tasks for ground equipment. KMZ files are 3D data in Google Earth and represent a map of the distribution of vegetation indices on satellite images. According to the results of experimental studies on the recognition of the values of vegetation indices, it was found that the data was distorted during the overlay of the images-the recorded colors were missing in the palette for the specified vegetation indices. Probable explanation in image correction for overlay on the satellite image to facilitate visual perception by the user. By comparing the data for the distribution points of the distribution map from the working window of the SlantView program, it was found that for the GeoTiff format color distortion and, consequently, the values of vegetation indices do not occur. Unlike the KMZ format, the file does not have positioning labels, but when you save the map, the program retains the scaling, and, accordingly, when using landmarks, the calculation of positioning is quite possible. In view of the above, the GeoTiff format was adopted for analysis.
Evaluation of the contour of the map: To manage the harvest, farms, regardless of weather conditions, need maps of the distribution of vegetation indices in many production fields available on the farm. Based on these circumstances, the Slantrange sensor complex was created to survey up to 10,000 ha/day, which can be provided on aircraft platforms. Since the average area of production fields in the plains of Ukraine is 70-100 hectares, it is desirable to survey several fields at once during one flight. In the analysis of a particular field, it is necessary to determine its boundaries. The Python-supported OpenCV library contains ready-made procedures for finding the contours of graphical objects that can be used in this case. An example of the result of card processing is shown in Fig. 1.
Fig 1. Green Chlorophyll index distribution map (a) for wheat crops affected by the after-effects of herbicides, (b) a photograph in pseudo-colors of the area highlighted by the square on the map, and (c) the results of the field contour search by the proposed software.
It should be noted that the forest-steppe zone of Ukraine is characterized by strong forest belts, the leaf cover of which, as well as the shadow from them are also fixed by the system. From the available experience, maintenance of forest belts and their renewal is not carried out regularly and there are many cases when tree crowns completely cover the road surface due to which significant errors are possible in determining the contours of crops. Since the field boundaries are stable, to analyze the presence of technological stresses on the maps of the distribution of vegetation indices stored in the Geotiff format, it is advisable to enter them manually, using certain reference points.
Estimation of the orientation of the sections of the field caused by stresses of technological character: With the identification of crop rows, the direction of crop rows was stable, but this is not a prerequisite for technological stresses. Thus, Fig. 1a shows the presence of a green band on the left and top, which for this index Green Chlorophyll index corresponds to healthier crops than those with yellow. This condition may be due to the best condition of mineral nutrition at the field boundaries because it is there that the equipment slows down, turns, adjusts the operation of nozzles, augers, and more. The width of such a layer, as a rule, does not exceed the radius of reversal of ground equipment, which can be taken into account when analyzing the distribution of stress areas.
The distribution in the field of stress areas caused by phytotoxic action (aftereffect) of herbicides, as well as violation of the seeding rate, is related to the direction of technological tracks, the organization of which meets certain rules. This is due to the fact that the introduction of chemical reagents or seeds during sowing is not carried out in an arbitrary manner, namely in compliance with the laid technological tracks. The directions of technological tracks in one field can change from year to year to maintain soil fertility, but their number usually does not exceed 2, in some cases 3 directions. Soil loosening can be carried out in any order, but technological stresses cannot be caused by this operation. Determining the direction of technological tracks has certain prerequisites, so mechanics when planning work are interested in the maximum length of the runs. Accordingly, in the absence of data from technological maps for the implementation of mechanical tillage, the orientation of the experimental field should be carried out along the maximum length of the field.
Fig. 2 presents a map of the distribution of stress areas for winter wheat crops where chemical poisoning of winter wheat crops as a result of the after-effects of herbicides from the predecessor crop was recorded.
Fig 2. Distribution maps of stress areas for the GreenNDVI index for winter wheat crops from April 27, 2019. Image of the Slantview software map window (left) and converted from a GeoTIFF file to jpeg format.
The specificity of SlantView software data processing is the observance of the north-south geographical orientation. As a result, to reduce the amount of data that does not belong to the field under study, it is necessary to change the orientation from geographical to local reference to the dimensions of the field. Due to the change in the orientation of the image, the number of pixels of the image obtained from the GeoTIFF file decreased from 1100 × 1660 to 245 × 1521, ie the amount of data decreased almost 5 times.
Fig. 3 shows the interface of the developed program in python to identify stressful areas of technological nature.
Fig 3. Picture of the program command-line interface when entering analysis parameters.
Convert data from color format to numeric view
Fig. 4 shows the map window palette, which is used to encode data and save them in tiff format according to the method presented in Shvorov, S. et al., (2020). Since the NDVI indices for plants change in the range 0…1 for visualization in the 8-bit color model, the index values were multiplied by 255 (Fig. 4).
Fig 4. The image of the Green NDVI map is listed from the Slantview software palette (encoding "shades of gray" where the index value is multiplied by 255.
To assess the presence of stress of a technological nature, the field image was divided into separate sections. The size of the plot was determined based on the resolution of the distribution map and the standard nomenclature of ground equipment available on the farm. The size was 13 × 13 pixels (6.5 × 6.5 m).
Calculation of distribution parameters
The GaussAmp equation was used to approximate the experimental data on the color intensity when color-coding the values of the intensity of the GreenNDVI index. Determined the value of the average value. Fig. 5 shows the image of the program interface when indicating the intermediate results for statistical processing of the distribution of index values on the map segment.
Fig 5. Image of the command line interface of the program during the program with the output of intermediate results of the analysis (in areas highlighted in green probable stress of a technological nature).
Filtering of the received data
Previous studies have found that stress should be determined by the magnitude of the standard deviation. For filtration, a limit value was set at which the stress status was set for the plots.
Graph analysis for in-depth search
The stressful state of plantations is caused by chemical poisoning of plants or their thickening due to non-compliance with production technology when moving ground equipment. Accordingly, stress areas will form bands. The DFS (Depth-first search) method was used to identify such stress areas. That is, single manifestations of plant stress due to differences from the total mass of the water supply regime, etc. are not taken into account. The results obtained are presented in Fig. 6.
Fig 6. Picture of the command line interface of the program during the output of the analysis results.
The developed software passed a production test, during which its accuracy and selectivity were confirmed.
High-resolution maps of high-resolution stress indices can be considered as a separate object of study on the interpretation of the causes of the stress of complex biological objects, such as winter crops. For remote monitoring of the state of winter crops, an algorithm for the identification of technological stresses has been developed on the basis of the spectral-spatial analysis of the nature of the location of stress areas. The algorithm is implemented in the developed software for spectral-spatial analysis of stress index maps to identify stress areas due to technological factors.
It has been experimentally confirmed in the production fields that the use of the developed software allows identifying the contours of areas of plants with stresses of a technological nature on the basis of stress index distribution maps.
Lysenko, V., Zhyltsov, A., Bolbot, I., Lendiel, T., Nalyvaiko, V. (2020). Phytomonitoring in the Phytometrics of the Plants. In E3S Web of Conferences, EDP Sciences, 154:07012.
Korobiichuk, I., Lysenko, V., Reshetiuk, V., Lendiel, T., Kamiński, M. (2016). Energy-efficient electrotechnical complex of greenhouses with regard to quality of vegetable production. Recent Advances in Systems, Control and Information Technology. Advances in Intelligent Systems and Computing, 543:243-251.
Tarariko, O. (2017). Long-term prediction of climate change impact on the productivity of grain crops in Ukraine using satellite data. Tarariko, O., Ilienko, T., Kuchma, T., Velychko, V. Agricultural Science and Practice. 4:3-13.
Lotfi, M., Solimani, A., Dargazany, A., Afzal, H., Bandarabadi, M. (2009). Combining wavelet transforms and neural networks for image classification. System Theory, Southeastern Symposium on, IEEE, pp:44-48.
Shvorov, S., Komarchuk, D., Pasichnyk, N., Opryshko, О., Gunchenko, Y., Kuznichenko, S. (2018). UAV navigation and management system based on the spectral portrait of terrain. International Conference on Methods and Systems of Navigation and Motion Control (MSNMC), Proceedings, рр:68-71.
Xianlong, Z., Fei, Z., Yaxiao, Q., Laifei, D., Xiaolong, W., Shengtian, Y. (2019). New research methods for vegetation information extraction based on visible light remote sensing images from an unmanned aerial vehicle (UAV). International Journal of Applied Earth Observation and Geoinformation, 78:215-226.
Myint, S.W., Gober, P., Brazel, A., Grossman-Clarke, S., Weng, Q. (2013). Per-pixel vs. Object-based classification of urban land cover extractio n using high spatial resolution imagery. Remote Sensing Environment, 115:1145-1161.
Andrea, S., Laliberte, E., Fredrickson, L., Albert, R. (2007). Combining decision trees with hierarchical object-oriented image analysis for mapping arid rangelands. Photogrammetric Engineering and Remote Sensing, 73:197-207.
Mitch, B., Alistair, R., Fabio, R., Salah, S. (2010). Airborne vision‐based mapping and classification of large farmland environments. Special Issue: Visual Mapping and Navigation Outdoors, 27:632-655.
Dorji, P., Fearns, P. (2017). Impact of the spatial resolution of satellite remote sensing sensors in the quantification of total suspended sediment concentration: A case study in turbid waters of Northern Western Australia. PLoS ONE, 12:e0175042.
Senthilnath, J., Kandukuri, M., Dokania, A., Ramesh, K.N. (2017). Application of UAV imaging platform for vegetation analysis based on spectral-spatial methods. Computers and Electronics in Agriculture, 140:8-24.
Komarchuk, D.S., Pasichnyk, N.A., Opryshko, O.O., Lysenko, V.P. (2021). Information technologies of remote assessment of herbicide consequences on winter rape crops. Scientific reports of NULES of Ukraine, 2:90.
Yu-Hsuan, Tu., Stuart, P., Kasper, J., Andrew, R., Dan, W. (2020). Optimizing drone flight planning for measuring horticultural tree crop structure. ISPRS Journal of Photogrammetry and Remote Sensing, 160:83-96.
Ming-Der, Y., Jayson, G., Boubin, H.P.T., Hsin-Hung, T., Yu-Chun, H., Christopher, C.S. (2020). Adaptive autonomous UAV scouting for rice lodging assessment using edge computing with deep learning EDANet. Computers and Electronics in Agriculture, 179:105817.
Wojciech, G., Edyta, P., Paweł, Ć., Wojciech, M. (2019). Application of convolutional neural networks for low vegetation filtering from data acquired by UAVs. ISPRS Journal of Photogrammetry and Remote Sensing, 158:1-10.
Smirnov, I., Kutyrev, A., Kiktev, N. (2021). Neural network for identifying apple fruits on the crown of a tree. International Scientific Forum on Computer and Energy Sciences (WFCES 2021), 270:01021.
Qi, Y., Liangsheng, S., Jinye, H., Yuanyuan, Z., Penghui, Z. (2019). Deep convolutional neural networks for rice grain yield estimation at the ripening stage using UAV-based remotely sensed images. Field Crops Research, 235:142-153.
Lysenko, V., Komarchuk, D., Opryshko, O., Pasichnyk, N., Zaets, N. (2017). Determination of the not uniformity of illumination in process monitoring of wheat crops by UAVs. 2017 4th International Scientific-Practical Conference Problems of Infocommunications Science and Technology, PIC S and T 2017 Proceedings, pp:265-267.
Yan, P., Yeyin, S., Shancheng, G., Feng, J., Arun-Narenthiran, V., Laura, T., Joe, L., Chao, L. (2020). Improved crop row detection with deep neural network for early-season maize stand count in UAV imagery. Computers and Electronics in Agriculture, 178:105766.
Junfeng, G., Wenzhi, L., David, N., Peter, L., Jürgen, V., Aleksandra, P., Yong, H., Jan, G.P. (2018). Fusion of pixel and object-based features for weed mapping using unmanned aerial vehicle imagery. International Journal of Applied Earth Observation and Geoinformation, 67:43-53.
Carlos, H.W.S., Rubens, A.C.L., Jansle, V.R., Paulo, S.G.M. (2017). Mapping skips in sugarcane fields using object-based analysis of unmanned aerial vehicle (UAV) images. Computers and Electronics in Agriculture, 143:49-56.
Emad, E., Martin, S., Kristian, H.T., Kjeld, J., Ulrik, P.S. (2018). A Survey of Open-Source UAV Flight Controllers and Flight Simulators. Micro-processors and Microsystems.
Shvorov, S., Lysenko, V., Pasichnyk, N., Rosamakha, Y., Rudenskyi, А., Lukin, V., Martsyfei, A. (2020). The method of determining the amount of yield based on the results of remote sensing obtained using UAV on the example of wheat. 2020 IEEE 15th International Conference on Advanced Trends in Radioelectronics, Telecommunications and Computer Engineering (TCSET), рр:245-248.
Khort, D., Kutyrev, A., Smirnov, I., Osypenko, V., Kiktev, N. (2020). Computer vision system for recognizing the coordinates location and ripeness of strawberries (Conference Paper). Communications in Computer and Information Science DSMP 2020, 1158:334-343.
Citation: Pasichnyk, N.A., Opryshko, O.O., Komarchuk, D.S. (2022). Spectral-spatial analysis of remote monitoring data to identify agrophytocenoses stresses of technological nature. Ukrainian Journal of Ecology. 12:6-12.
Received: 20-Dec-2021, Manuscript No. UJE-21-50280; Accepted: 21-Jan-2022, Pre QC No. P-50280; Editor assigned: 22-Dec-2021, Pre QC No. P-50280; Reviewed: 05-Jan-2022, QC No. Q-50280; Revised: 12-Jan-2022, Manuscript No. R-50280; Published: 01-Feb-2022, DOI: 10.15421/2022_328
Copyright: This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.