Cheryl Porter, University of Florida
Sander Janssen, Alterra Wageningen UR
The goals of the AgMIP IT team are to facilitate the compilation, archiving, and exchange of data and information for the AgMIP research community and stakeholders.
The main objectives of the team are:
1) Develop an IT infrastructure for the AgMIP project that allows easy and secure access to shared data, models and results of researchers in the AgMIP consortium, with both a short and long term perspective;
2) Facilitate primarily the use of data by models and exchange of model results and secondly, the linking of models relevant for the AgMIP project to allow for reproducible and repeatable applications;
3) Explore the potential of state-of-the-art Information & Communications Technologies, relevant to improve modeling with a long-term perspective, including web-based model executions and Service Oriented Architecture (Bio-informatics);
4) Organize the online dissemination of AgMIP data and outputs.
At the core of AgMIP is the use of multiple models for the purposes of quantifying model uncertainties and improving the performance of all participating models. The schematic diagram above shows the general categories of tools that are envisioned for AgMIP to enable the multiple climate, crop, and economic models to be used for model intercomparisons and assessments.
One of the goals of the AgMIP project is a comparison of the simulated results from various crop models for the purposes of quantifying model uncertainties and as a basis for model improvement. Multiple crop models are used to generate simulated estimates of crop yields and other data using identical input data for each model. But the format of the input data varies among the models. Some models use text file inputs, some use XML files, some use Excel spreadsheets, etc. Manual translation of data into the various model formats is not practical for large amounts of data that will be processed by AgMIP researchers.
To solve the problem, AgMIP will store crop experiment data in a harmonized data format and this harmonized data will be exported to the formats specific to each model. We are in the process of developing the data translation tools to create the input data specific to each model. The AgMIP IT team is working with the crop model development teams for each crop model to accomplish this goal. A series of development workshops are being organized that focus on programming the applications necessary for exporting data from the AgMIP harmonized crop experiment database to the data formats specific to each of the major crop models associated with AgMIP.
The first AgMIP IT development workshop was held at the South Asia Regional workshop in India, February 20-24, 2012. This workshop began the development of data translation tools for Aquacrop, Infocrop, ORYZA and STICS. A tool for importing data from a spreadsheet format was also developed.
The second development workshop was held in Gainesville, Florida, July 9-14, 2012. This workshop concentrated on developing data translation tools for APSIM, Aquacrop, STICS and WOFOST crop models. Many discussions were held during the course of the week that allowed us to improve the harmonized data structure to accommodate a wider range of modeling perspectives. Some of these changes slowed development of existing translators, but allowed for a much better overall system for development for these models and future AgMIP developers.
A third development sprint workshop was held at the AgMIP North America Regional workshop in Ames, Iowa September 4-7, 2012 which included EPIC, CropSyst, Salus and RZWQM2 models. Further enhancements to the data structures and modeling environment resulted from this workshop.
The 4th development sprint (AKA The Austin Hackathon), hosted by the iPlant Collaborative, was held at the Texas Advanced Computing Center in Austin, TX, USA, February 4-8, 2013. Participants at this workshop developed data translation tools for APSIM, AquaCrop, CropSyst, InfoCrop, SarraH, STICS and WOFOST.
Tools and Applications
AgMIP tools and applications are being developed collaboratively using a github repository (http://github.com/agmip). A wiki describing AgMIP tool development protocols is hosted at the AgMIP IT team wiki. The wiki shows information about our development environment, coding standards, and the GIT version control system. AgMIP applications developers can sign up for a GitHub account and fork one of the AgMIP repositories and begin collaboration immediately.
Crop model input data from (field experiment sentinel sites) comprise just one category of data that will be used in AgMIP. Other data types, as listed in Figure 1, are climate scenarios, generalized soil properties, GIS raster and vector data, Representative Agricultural Pathways (RAPs) and crop scenarios, crop model output data, regional economic model inputs and outputs, and global economic model outputs. Each of these categories of data will be associated with a harmonized database and the data translation tools appropriate for the associated models.
AgMIP Data Services
AgMIP data will be stored in a distributed system of data nodes using a synchronized Riak database structure. A centralized metadata store will allow synchronization and exposure of metadata for searching all available data nodes. In this way, a data node in South Asia may contain regional data, but a researcher in South America may search and download data as needed using the AgMIP web data portal.
Highlights of the 2011-2012 AgMIP IT Team can be downloaded as a pdf here.