CAREER: Scalable Remote Sensing Computational Framework for Near-real-time Crop Characterization

The increasing proliferation of earth observation satellites, along with the explosive growth of remote sensing data, has dramatically facilitated timely land surface characterization worldwide. With several national and international agricultural initiatives, near-real-time crop type characterization has become vital for providing early warnings on food insecurity and timely crop yield forecasting, and for global food market transparency. However, near-real-time crop type characterization remains a challenge in agricultural remote sensing, due to the difficulty in collecting timely crop ground reference data, the limited generalizability of existing characterization models, and the lack of appropriate remote sensing cyberinfrastructure. Recent advances in satellite remote sensing and computational cyberinfrastructure open a new avenue to tackle the challenge. The overarching goal of the project is to establish a scalable remote sensing computational framework for near-real-time crop type characterization and to promote computational remote sensing education. The computational framework can transform the large-scale agricultural monitoring paradigm to meet the timely crop characterization requirements of global agricultural initiatives. The framework can substantially boost the ability to respond rapidly to emerging food crises, as well as create cross-cutting impacts in advancing a broad spectrum of remote sensing and agricultural research. The synergistic education and outreach activities offer unique learning opportunities about computational remote sensing to students from K-12 to the graduate level, and will broaden the participation of underrepresented students in computing. These activities also facilitate the open development and adoption of the computational framework across a range of disciplines. Therefore, this research aligns with the NSF mission to promote the progress of science and to advance the national health, prosperity, and welfare.

The advanced remote sensing computational framework focuses on the development of a benchmark data repository called CropSight, a crop characterization modeling system, and a cutting-edge remote sensing cyberinfrastructure, to catalyze near-real-time crop and land surface characterizations. CropSight is a unique national-scale crop ground reference data repository, and embodies a wealth of season-long remotely sensed crop growth and environmental attributes across crop growing locations for most crop types in the U.S. CropSight can be generalized to continental and global scales, and will be used as a large-scale, systematic, and consistent ground reference data repository. The crop characterization system comprises a suite of novel deep learning-based computational models that can fuse the imagery from a set of earth observation satellites for timely crop monitoring, as well as identify varying crop types via innovative modeling of complex crop-environment interactions. The system will increase modeling generalizability for crop type characterization, and holds considerable potential to be extrapolated over wide geographical regions. The remote sensing cyberinfrastructure will include a highly scalable and cloud native implementation of the CropSight and a near-real-time on-demand crop monitoring system. With a serverless architecture, the project will build the cloud middleware that integrates various geospatial data sources and enables data-intensive remote sensing data analytics for timely crop characterization. The cyberinfrastructure will empower the paradigm shift from conventional compute-limited remote sensing analysis to massive imagery analysis for timely land surface monitoring.

Publications:

Diao, C., and G. Li. (2022). Near-surface and high-resolution satellite time series for detecting crop phenology. Remote Sensing, 14(9), 1957. https://doi.org/10.3390/rs14091957

Yang, Z., Diao, C. and F. Gao. (2023). Towards scalable within-season crop mapping with phenology normalization and deep learning. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,16, 1390-1402. https://doi.org/10.1109/JSTARS.2023.3237500

Zhang, C. and C. Diao. (2023). A Phenology-guided Bayesian-CNN (PB-CNN) framework for soybean yield estimation and uncertainty analysis. ISPRS Journal of Photogrammetry and Remote Sensing, 205, 50-73. https://doi.org/10.1016/j.isprsjprs.2023.09.025

Liu, Y., Diao, C. and Z. Yang. (2023). CropSow: an integrative remotely sensed crop modeling framework for field-level crop planting date estimation. ISPRS Journal of Photogrammetry and Remote Sensing, 202, 334-355. https://doi.org/10.1016/j.isprsjprs.2023.06.012.

Liu, Y., Diao, C., Mei, W. and C. Zhang. (Under review). CropSight: towards a large-scale operational framework for object-based crop type ground truth retrieval using street view and PlanetScope satellite imagery. ISPRS Journal of Photogrammetry and Remote Sensing

Yang, Z., Diao, C. and F. Gao (Under revision). EMET: A emergence-based thermal phenological framework for near real-time crop type mapping. ISPRS Journal of Photogrammetry and Remote Sensing

Zhou, Z., Liu, Y., and C. Diao (Under review). National-scale object-based crop type ground truth data repository using street view and satellite imagery. Earth System Science Data

 

Conference Presentations:

Diao, C., Yang, Z., and Y. Liu. Monitoring diverse crop phenological stages using satellite time series. Annual Meeting of the Association of American Geographers (AAG), Honolulu, HI. April 16-April 20, 2024.

Yang, Z. and C. Diao. A novel deep learning framework for within-season field-level crop phenology characterization. Annual Meeting of the Association of American Geographers (AAG), Honolulu, HI. April 16-April 20, 2024.

Zhou, Z., Liu, Y., and C. Diao. CropSight: towards a national-scale operational framework for object-based crop type ground truth retrieval using street view and PlanetScope satellite imagery. Annual Meeting of the Association of American Geographers (AAG), Honolulu, HI. April 16-April 20, 2024.

Chen, J., Diao, C., Liu, Y., Yang, Z., and Z. Zhou. CropSync: towards a large-scale operational framework for within-season crop type mapping using Google Street View and Harmonized Landsat and Sentinel-2 imagery. Annual Meeting of the Association of American Geographers (AAG), Honolulu, HI. April 16-April 20, 2024.

Diao, C., Yang, Z., Liu, Y., Zhang, C. and T. Guo. Towards large-scale crop phenological characterization using multi-scale satellite time series. American Geophysical Union (AGU) Fall Meeting, San Francisco, CA, December 11-December 15, 2023. (Invited Talk).

Yang, Z., Diao, C., and F. Gao. EMET: A emergence-based thermal phenological framework for near real-time crop type mapping. American Geophysical Union (AGU) Fall Meeting, San Francisco, CA, December 11-December 15, 2023.

Zhang, C. and C. Diao. Enhancing crop yield prediction across regions with phenology-based domain adaptation: a meta-learned conditional adversarial approach. American Geophysical Union (AGU) Fall Meeting, San Francisco, CA, December 11-December 15, 2023.

Yang, Z. and C. Diao. A novel deep learning framework for within-season field-level crop phenology characterization. American Geophysical Union (AGU) Fall Meeting, San Francisco, CA, December 11-December 15, 2023.

Liu, Y., Diao, C., Mei, W. and C. Zhang. CropSight: an operational framework for crop type information retrieval using street view and PlanetScope satellite images. American Geophysical Union (AGU) Fall Meeting, San Francisco, CA, December 11-December 15, 2023.

Guo, T., Diao, C., Zhang, C., Liu, Y. and Z. Yang. Towards scalable field-level crop yield estimation through integration of crop model and deep learning. American Geophysical Union (AGU) Fall Meeting, San Francisco, CA, December 11-December 15, 2023.

Diao, C. Development of large-scale crop phenological characterization framework with satellite time series. Annual Meeting of the Association of American Geographers (AAG), Denver, CO. March 23-March 27, 2023.

Guo, T. and C. Diao. Towards scalable field-level crop yield estimation through integration of crop model and deep learning. Annual Meeting of the Association of American Geographers (AAG), Denver, CO. March 23-March 27, 2023. (Student Honors Paper Competition Award).

Yang, Z. and C. Diao. Within-season crop mapping at the field level using a phenology-guided deep learning model. Annual Meeting of the Association of American Geographers (AAG), Denver, CO. March 23-March 27, 2023.

Diao, C. and G. Li. Monitoring crop phenology with near-surface and high-resolution satellite time series. American Geophysical Union (AGU) Fall Meeting, Chicago, IL, December 12-December 16, 2022.

Zhang, C. and C. Diao. A novel phenology-guided Bayesian-CNN framework for crop yield estimation. American Geophysical Union (AGU) Fall Meeting, Chicago, IL, December 12-December 16, 2022.

Liu, Y., Diao, C., Yang, Z., and E. Nafziger. CropSow: an integrative remotely sensed crop modeling framework for field-level crop planting data estimation. American Geophysical Union (AGU) Fall Meeting, Chicago, IL, December 12-December 16, 2022.

Yang, Z., Diao, C., and F. Gao. A novel phenology guided deep learning model for within-season field-level crop mapping. American Geophysical Union (AGU) Fall Meeting, Chicago, IL, December 12-December 16, 2022.

Yang, Z. and C. Diao. A phenology-guided deep learning model for early crop mapping at the field level. Annual Meeting of the Association of American Geographers (AAG), New York City, NY. February 25-March 1, 2022.

Diao, C. Towards remote sensing modeling framework for crop phenological characterization. Annual Meeting of the Association of American Geographers (AAG), New York City, NY. February 25-March 1, 2022.

Diao, C., Yang, Z., Gao, F., Zhang, X. and Z. Yang. A novel hybrid phenology matching model for robust crop growth stage characterization. American Geophysical Union (AGU) Fall Meeting, New Orleans, LA, December 13-December 17, 2021.

 

Conference Sessions:

“Advances in Multitemporal Remote Sensing for Terrestrial Ecosystems”, Annual Meeting of the Association of American Geographers (AAG), Honolulu, HI. April 16-April 20, 2024.

“Advances in Agricultural Remote Sensing and Artificial Intelligence”, Annual Meeting of the Association of American Geographers (AAG), Denver, CO. March 23-March 27, 2023.

"Advances in Agricultural Remote Sensing and Artificial Intelligence", Annual Meeting of the Association of American Geographers (AAG), New York City, NY. February 25-March 1, 2022.