Breeding Information Management System (BIMS)

Body: 

CGD BIMS (Breeding Information Management System) is a Tripal module being developed by Mainlab Bioinformatics which allows users to explore the publically available breeding data in CGD while also providing a data management solution for private breeding programs.  BIMS is now open for breeders to create an account and start using it with current functionality. It is being continuously developed and once complete it will provide individual breeders with a secure and comprehensive online breeding management system that allows them to store, manage and analyze their private data, fully integrated with publicly available genomic, genetic and breeding data within Tripal databases such as CGD.

The BIMS tool now supports the use of Field Book, an Android app developed by the Poland Lab at Kansas State University for collecting phenotype data in the field.  Field Book is a free app that is available through the Google Play Store.  It provides easy and customizable interfaces for entering all types of biological data.  For more details about Field Book, please visit the developer's website to view the current manual and for more information. In the following sections we provide information on how to use the current version of BIMS, which will be updated as more functionilty becomes available. If you would like to be kept informed on progress please sign up for the CGD BIMS mailing list. The links below provide information on how to access CGD BIMS, View Data, Search Data, Create a Breeding Program, Import Data, Manage Data using Field Book and Archive Data. If you have any questions please contact using the contact form on CGD. We will be holding monthly "how to webinars" and are happy to provide individual program training and guidance and solicit input on further development. 

To access BIMS, simply click on the "Tools" menu (Fig. 1A) and then select "BIMS" (Fig. 1B).  The BIMS interface will then open.  Public data is available for all users of CGD without requiring an account on CDG.  Users can download public data from BIMS without an account, but will need their CGD user account to save the results. Access to private breeding data requires users to login with their CGD user account.  To create a breeding program, users need to create a CGD account and contact using the contact form for breeder privilege.

BIMS: Viewing Data

Body: 

To view data that has already been loaded into BIMS, you first need to choose the breeding program (Fig. 2A).  Once the selections have been made, the selected program names will appear in the upper left corner of the BIMS window  (Fig. 3A).  You can change the program by clicking on the buttons in the upper left corner.

Figure 2.  Selecting the Program in BIMS.

There are three main areas of the BIMS interface.  The first part is the header region (Fig. 3A) which displays the current program, allows users to change program, has a link to the CGD homepage (Site Home), and displays the user name.  The second part of the interface is the accordion menu on the left side (Fig. 3B).  This menu allows users to switch between different sections of the BIMS program.  The last region is the tab region (Fig. 3C) and this is where users primarily interact with BIMS.  Each tab in BIMS has an "Instructions" section that can be expanded by clicking on it.  The "Instructions" have information on how to use that tab.

Figure 3.  The main parts of the BIMS interface.

Once the program has been loaded, you can explore the data that have been loaded into BIMS by clicking on the "Manage Breeding" section of the accordion menu.  Under "Manage Breeding" there are links to view information about the Program, Trait, Location, Cross, Accession, and Trial.  To view information about a certain trait, click on "Trait" (Fig. 4A) and the "Manage Trait" tab will appear.  In that tab, click on the trait of interest (Fig. 4B).  Once a trait is selected, the "Trait Details" will be displayed (Fig. 4C).

Figure 4.  Selecting and viewing Trait Details.

If you are viewing data from a private breeding program, you can also edit the information for all the items under manage breeding.  Breeders with private breeding programs will also see an "Actions" section (Fig. 5A) on the details sections for the trait, cross, etc.  The "Actions" section allows breeders to edit the properties or delete the item.

Figure 5.  Editing abilities available to breeders with private programs.

You can also view information about the other data types under "Manage Breeding" by clicking on each type.  For "Location," information about each location can be displayed.  First select "Location" (Fig. 6A), then choose the location name (Fig. 6B), and then the details will appear (Fig. 6C).

Figure 6.  Location details.

For the "Cross" (Fig. 7), "Accession" (Fig. 8), and "Trial" (Fig. 9) data types, the data details are accessed the same way.  First select the data type under "Manage Breeding" (Fig. 7A-9A), then select the one of the names (Fig. 7B-9B), and then the details will appear (Fig. 7C-9C).  For these data types, in the "Details" section there is also a "View" button that opens another tab that displays further statistics/data.

Figure 7.  Accession details.

Figure 8.  Accession details.

Figure 9.  Trial details.

BIMS: Searching Data

Body: 

You can also search the data in a breeding program using BIMS.  Click on the "Search" section of the left side accordion menu and then click on "Accession" (Fig. 10A).  This will open the "Search Accession" tab (Fig. 10B). 

Figure 10.  Opening the Search tab.

The first step to start a search is to select a property or trait under the "Choose property" section (Fig. 11A).  In this example, we are choosing "Parent."  After the property is selected, the "Set filter" section will populate with options for that data type (Fig. 11B).  After selecting the options you want, click "Add."  The selected filter will then appear in the "Search results" section (Fig. 11).  If you want to reset the search, click the "Reset" button (Fig. 11C).

Figure 11.  Selecting a Property and setting a Filter in the Search Tab.

You can further filter the search results by adding another property or trait under the "Choose property" section.  In this example, we are further refining the search by adding the Fruit Shape trait (Fig. 12A).  The "Set filter" section will show the data that are relevant for the previous filter you chose. For example, the statistics shown for Asian Citrus Psyllid Adult Mean trait here is only for those accession from the trial you chose in the previous search. You can set the minimun and maximum value of this trait, and then click "Add" (Fig. 12B).  The new filter is added to the "Search results" section and the selected filters can be removed with the "Remove" button under "Search results" (Fig. 12C).  To view the search results, and to generate a file to export, click on "View" under "Search results" (Fig. 12C).

Figure 12.  Adding more filters to the search and viewing results.

When the "View" button is clicked (Fig. 12C), a new tab opens called "Search Results" (Fig. 13).  On this tab there is a "Column options" section (Fig. 13A) and a preview of the data table (Fig. 13B).

Figure 13.  Search Results tab.

Users can add more information to the table before exporting the data.  To do this, click on the "Column options" section to expand it (Fig. 14).  There are two sections, "Properties" (Fig. 14A) and "Traits" (Fig. 14B).  You can select additional items to display by clicking on the check boxes in these two sections.  Once all the selections have been made, click "Update" (Fig. 14C).

Figure 14. Adding more columns to search results table with Column Options section.

After selecting additional data to display on the table, you can preview the table on the "Search Results" tab (Fig. 15A) and see the original filters that were used to generate the initial table by clicking on "Descriptions" in the "Results" section (Fig. 15B).  To download the data table, click on the "Download" button in the "Save Results / Download Data" section (Fig. 15C).  If you do not see your browser start to downoad the file after clicking "Download," please make sure the pop-up was not blocked.

Figure 15.  Downloading data from the Search Results tab.

BIMS generates a CSV file of the search results table (Fig. 16).   This table is easily opened by Microsoft Excel and other programs.

Figure 16.  Example CSV file exported from BIMS.

If you have a CGD account, you can also save the search results to a list.  You will see a "Save" button with boxes for a list name and description (Fig. 17A).  This list will be saved to your program or to public program that you are in and accessed by clicking on "Search Results" in the "Search" section of the accordion menu (Fig. 17B).  You can also see details about the saved lists and delete, view, or edit them (Fig. 17C).

Figure 17.  Saving lists from searches to private breeding programs.

BIMS: Creating a New Breeding Program

Body: 

To start a new breeding program in BIMS, you must have a CGD account with the appropriate access level.  The first step in creating a new program is to select the Program Button (Fig. 18A). After selecting a program, the "Manage Program" tab will open and then you can open the "Program Admin Menu" section (Fig.18B).  Click "Create" to make a new program.

Figure 18.  Switching crops and opening the Manage Program tab.

After clicking the "Create" button, the "Add Program" tab opens (Fig. 19A).  Enter the new Program Name and make sure that the required column names are set to match your dataset (Fig. 19B).  The four columns are required columns in phenotype sheet of the data template and if you change the column names here the column names in the phenotype sheet of the template for download will automatically change. If you use Field Book App for phenotype collection, use these column names in the input files. After you have created the program, you will have the option to add other CGD users as members of the breeding program (Fig. 19C).  Once the program is created, data can be loaded to the program.

Figure 19.  Creating a new program and adding members.

BIMS: Importing Data

Body: 

To import data into BIMS, the data need to be in BIMS templates. Phenotype data can either needs to be files exported from Field Book or data entered into the BIMS templates.  To download the BIMS templates, click on the "Data Import" section of the left-hand accordion menu and then click on "Template List" (Fig. 20A).  You will then see the "Data Templates" tab that has information about each template and links to download the template (Fig. 20B).  You can also download all the templates as a single Excel file by clicking below (Fig. 20C).

Figure 20.  Downloading BIMS data templates. 

To upload data files, click on the "Upload data" link (Fig. 21A) under the "Data Import" section and the "Upload Data" tab will open.  Then, under the "Upload data by Excel template" section on the right side of the "Upload Data" tab, select the data file and click "Upload" (Fig. 21B).  Enter a file description if desired, and once the file has uploaded, click "Submit".  The file name will then appear in the "Uploaded Files" section (Fig. 21C).

Figure 21.  Upload Data tab in BIMS.

To view the progress of an upload job, click on the file name under "Uploaded files" (Fig. 22A).  Summary information about the file will appear below and then click the "View" button (Fig. 22B).  The "Upload Job" tab appears and will show the "Job Details" table (Fig. 22C).  On the table you can see the job progress.  If the job is still running, you can click the "Refresh" button to update the job progress.

Figure 22.  How to view upload job progress.

BIMS also provides links to log files on the "Upload Job" tab.  When a job successfully loads, there are three types of logs: New Data Logs, Warning Logs, and Duplicate Logs.  To view the logs, first expand the log type section you want to view and then click on "view" (Fig. 23A) to see the log messages (Fig. 23B).  New Data Logs describe the new data that was loaded.  Warning Logs indicate if any data was skipped due to incorrect column headers or typos, but this data was not critical for BIMS to load data so the job still completed.  It is recommended that you look at the Warning Logs to make sure data was not omitted accidently.  The Duplicate Logs display what data was a duplication of data already present in BIMS.

Figure 23.  BIMS upload job logs.

But what happens if there is an error in the data that causes the upload job to fail?  If an upload job fails, you will see "FAILED" under the job progress (Fig. 24A).   An Error Log section will also be available on the "Uploading Job" tab.  Expand the Error Log section and click on "view" (Fig. 24B) to see the error messages (Fig. 24C).  Use the Error Log information to correct the original file.

Figure 24.  Failed jobs and the Error Logs

To re-run a failed job, use the "Re-Run Job" section on the "Uploading Job" tab.  Select and upload the revised file (Fig. 25A) and then click the "Re-Run the Job" button (Fig. 25B).  To monitor the re-run progress, click on the "Refresh" button (Fig. 25C). If the error was something that you did online not in the template, for example you had a typo in the configuration setting, it is best not to use Re-Run Job, but go back and start the loading process again.

Figure 25.  Re-running a job.

BIMS: Field Book Management

Body: 

For breeders using the Field Book App, BIMS can help generate the trait and field input files to configure Field Book for data entry and BIMS can also import data in the trait and field files from Field Book.  For ongoing trials, BIMS can also help breeders sync data between different devices by loading the exported data from multiple devices into BIMS and generating an input file with combined information.  When breeders first create their programs, they have chosen the names for the four required columns. These columns should match the column names in Field Book Input file, and breeders make sure they match and change if needed. The configuration settings can be accessed either by clicking on "Configuration" under the "Field Book Management" section on the left-hand accordion menu (Fig. 26A) or by clicking on the icon in the header (Fig. 26B).  On the "Configuration" tab (Fig. 26C) there are two sections; "Field Book required columns" and "Field Book custom columns".  The first section is to set names for Accession and the three required columns in Field Book Input file, the second section is for users to choose the optional columns that they want to add in the Field Book Input file. When they generate Field Book Input file, these optional columns will show up for breeders to choose for each Field Book Input file that they generate.

Figure 26. Field Book configuration options.

Two different files types for input into Field Book can be generated using BIMS.  The first file type is the Trait (.trt) file when breeders want to add traits to Field Book using a file instead of creating one by one in Field Book App.  This file contains the details on the phenotypic traits being recorded.  To generate this file, click on "Trait" (Fig. 27A) and the "Trait" tab will open.  On the "Trait" tab, there will be a list of traits that have already been loaded into the program on BIMS (Fig. 27B).  Select the traits you want to export and then click the "Generate" button (Fig. 27C).  A .trt file will then download.

Figure 27.  Generating Trait file for Field Book.

The Field file (.csv) can also be generated for import into Field Book.  This file has the list of accessions and columns for the required columns (unique identifier, primary order, secondary order).  The Field file is generated by clicking on "Field" (Fig. 28A) to open the "Field" tab.  On the "Field" tab there are three options for generating the file (Fig. 28B).  The first option is to generate a "New Trial".  This can be used when breeders want to do another trail with accessions that are already in the database. If you have saved an accession list from a previous search (see Searching Data section), the option to use that list is available (Fig. 28C).  You can also select other accession properties to add to the file.  To generate the file, click the "Generate" button (Fig. 28D) and the file will download. In the generated file from this "New Trial" option, only the accession is provided and the three required colums (the unique identifier, primary order, and secondary order) will be empty since they depend on the planting design. The breeders should fill in those columns before importing into Field Book App.

Figure 28.  Generating a Field file for a New Trial.

Field files can also be generated from an Existing Trial.  Files from Existing Trials (Fig. 29A) contain the accessions from the existing trial that is selected as well as the custom columns the user chooses. This functionality is to provide a way to sync data from multiple devices that have Field Book App to check which individual has been phenotyped already. Breeders can first upload the data from multiple devices, then re-generate an input file from the existing trial with a trait value.

Cross option (Fig. 30) allows you to generate input file for a new cross. When users upload a file (Fig. 30A) with an information on the cross and the planting (eg. cross name, primary order, and progeny in each primary order) and type in prefix for the unique id (Fig. 30B), then BIMS generates a Field Book input file with new progeny names and three required columns automatically filled in (Fig. 30C). In the uploaded file (Fig. 30A), the order of the columns is important, not the colum head.

Figure 29.  Generating a Field file from an Existing Trial or Cross.

 

Figure 30.  Example input file and output file for Generating a Field file for a new Cross.

If you have already configured Field Book and would like to transfer the Trait and Field information to BIMS, that can be done by importing the Trait and Field files from Field Book.  To import a Trait file, use the "Upload trait file" option (Fig. 31A).  Select your file (Fig. 31B) and click "Upload."  Once the file has uploaded, click "Submit" (Fig. 31C) to start the import job.  The file then appears in the "Uploaded files" section (Fig. 31D).

Figure 31.  Importing Trait files from Field Book into BIMS.

Field files can also be imported into BIMS from Field Book.  Click on "Upload field file" (Fig. 32A) and then upload the field file and click "Inspect" (Fig. 31B).  After BIMS checks to see if the required columns are present in the Field file, the "Upload an exported field file" section appears (Fig. 31C) and the location and trial can be selected.  The file upload job is then initiated by clicking "Submit".

Figure 32.  Importing Field files from Field Book into BIMS.

 

BIMS: Archiving Data

Body: 

Creating an archive file of all the data in a breeding program is easily done using the "Archive" function in BIMS.  Open the "Archive" section in the left-hand accordion menu, and then click "Manage Archive" (Fig. 33A).  On the "Manage Archive" tab, there is an "Archive Data" button (Fig. 33B).  When this button is clicked, an Excel file is generated that contains all the data in the current breeding program (Fig. 33C).

Figure 33.  Generating an archive file.

 

BIMS: FAQ and Troubleshooting

Body: 

How do I search data?

How do I view/edit data in a program?

How do I import data?

How do I manage Field Book trait and field files with BIMS?

How do I create a new breeding program?

Troubleshooting

Section Problem Solution
General I have selected a crop and program, but do not see any options under Manage Breeding and other BIMS sections. Please refresh the page in your internet browser.
  I still have problems with viewing BIMS information, even after refreshing the page. Please clear the cached images/files, cookies, and hosted app data from your internet browser's history.  You will have to log back into BIMS and reopen your program.
Data Import/Field Book Management My data upload failed. Please make sure that the upload didn't fail due to data errors or typos first by looking at the error logs (see Importing Data section for more details).  If you need help, contact us.
  Even after I changed the max/min value of the trait descriptor in the template, BIMS still give an error saying that I have data outside the acceptable range. Once your descriptor has been loaded, you have to edit the the descriptor data in BIMS, not in template (go to Manage Breeding, Trait, choose the trait and click 'Edit' .)
 

What do the errors below mean?

The required column (unique_id) is missing
The required column (primary_order) is missing
The required column (secondary_order) is missing

BIMS allows you to change the column names in the template for 'accession_id', 'unique_id', 'primary_order', and 'secondary_order'. You get this type of error when the column names in your template do not match the ones in BIMS. Once you change it in BIMS, the templates downloaded from BIMS also changes. Please refer to Figure 26B in the manual.
  The output from Field Book Management - Generate input file (Field) - Cross option doesn't seem correct. Please check your file if it's in the order of cross name, row and the number of progeny for each row that you will plant. In this file, the order of the columns is important, not the column head. Please refer to Figure 30A in the manual.
Search I can't download my file. Please make sure that you allow pop-ups on your browser. Please refer to how to allow pop ups on chromeon firefoxon internet explorer.