Merge branch '7.x-3.x' into 525-tv3-obo_speed_colon

docs/user_guide/creating_content.rst

@@ -38,26 +38,65 @@ Alternatively, you can add terms one at a time. To add a single term either from
 Create a Tripal Content Type
 ----------------------------
 
-Creation of a new content type requires familiarity with Chado.  This is because data records used by content types must be mapped to actual data and the data lives in Chado.  Tripal's interface for creating content types allows you to provide the CV term for the type and then indicate where in Chado the data is/will be stored.  Chado is a flexible relational database schema.  Thus, it is possible for different sites to store data in different ways.  It is best practice however to follow community standards when storing data.  Therefore, please review the online documentation for Chado. If you are unclear how data for your content type should be stored in Chado please consider emailing the `Chado mailing list <http://gmod.org/wiki/GMOD_Mailing_Lists>`_ to ask for help or add a request for help on the Tripal issue queue.
+Creation of a new content type requires familiarity with Chado.  This is because data records used by content types must be mapped to actual data and the data lives in Chado.  Tripal's interface for creating content types allows you to provide the CV term for the type and then indicate where in Chado the data is/will be stored.  Chado is a flexible relational database schema.  Thus, it is possible for different sites to store data in different ways.  It is best practice however to follow community standards when storing data.  Therefore, please review the online documentation for Chado. If you are unclear how data for your content type should be stored in Chado please consider emailing the `Chado mailing list <http://gmod.org/wiki/GMOD_Mailing_Lists>`_ or posting an issue on the `Chado GitHub issue queue <https://github.com/GMOD/Chado>`_ to ask for help or add a request for help on the Tripal issue queue.
 
-To add a new content type, start by navigating to **Structure → Tripal Content Types** and  click on the **Add Tripal Content Type** link at the top. This will take you to a web form that leads you through the process of creating a custom Tripal Content Type. First, enter for the name of the term you would like to use to describe your content in the Content Type autocomplete textbox (e.g. genetic_marker). Then, click **Lookup Term**. This should bring up a list of matching terms from which you can select the specific term you would like to use.  Sometimes the same term exists in multiple vocabularies and you can select the proper one.
+To add a new content type, start by navigating to **Structure → Tripal Content Types** and  click on the **Add Tripal Content Type** link at the top. This will take you to a web form that leads you through the process of creating a custom Tripal Content Type.
+
+Genetic Marker Example
+^^^^^^^^^^^^^^^^^^^^^^
+To demonstrate how to create a new content type we will use the example of a genetic marker. First, enter for the name of the term you would like to use to describe your content in the Content Type autocomplete textbox. Then, click **Lookup Term**. This should bring up a list of matching terms from which you can select the specific term you would like to use.  Sometimes the same term exists in multiple vocabularies and you can select the proper one.
 
 .. image:: creating_content.create1.png
 
-During content type creation there is as a section to specify which Chado tables will store your data. Chado is typically structured with primary **base** tables (e.g. organism, feature, stock, project, etc) and a set of linker and property tables that contain ancillary data related to the base records.  Here you must first choose the base table where primary records for your data type are stored.  For our example, because genetic markers are sequence features, they are stored in the Chado feature table. Once you select the Chado table, the form will ask additional questions to determine exactly how records of the content type can be found. Specifically, the following options are supported if the appropriate fields/tables are available:
+During content type creation there is as a section to specify which Chado tables will store your data. Chado is typically structured with primary **base** tables (e.g. organism, feature, stock, project, etc) and a set of linker and property tables that contain ancillary data related to the base records.  Here you must first choose the base table where primary records for your data type are stored.  For our example, because genetic markers are sequence features, they are stored in the Chado **feature** table.
 
-1. All records in the **base** table belong to the content type (e.g. tables: organism, analysis, etc.)
-2. The **base** table has a **type_id** that stores the CV term and this differentiates the records. (e.g. tables: feature, stock, library, etc.).
-3. The records can be differentiated by way of a property table which contains a **type_id** column for the CV term. (e.g. tables: featureprop, stockprop, libraryprop, etc.)
-4. The records can be differentiated by way of a linking table that associates records in the **base** table with the CV term (e.g. tables: feature_cvterm, stock_cvterm, etc.)
+.. image:: creating_content.create2.png
 
-For our genetic marker example, we can use the Chado **feature** table and **type_id** column to differentiate which records in the feature table are genetic markers. Thus we
+Next, you will be asked if all of the records in the selected table are of the desired content type.  Usually the answer to this is "No", especially if the Chado table has a **type_id** column.  In our case. The **feature** table does have a **type_id** column so we must select "No".
 
-- Select "No" not all records in the feature table are genetic markers
-- Type Column: type_id
+.. image:: creating_content.create3.png
 
-Then click Create Content Type to create a custom genetic marker content type.
+If all of the records in the selected table do not belong to the content type then the form knows enough about each Chado table to offer you appropriate options for how to store data for your content type.  The form knows that the **feature** table has a **type_id** column so it asks if we can differentiate records for our content type using the **type_id** field.  For our example genetic marker we can do so.
 
-.. image:: creating_content.create2.png
+.. image:: creating_content.create4.png
+
+Just prior to creating our content type we are provided a summary of our selection to review:
+
+
+.. image:: creating_content.create5.png
+
+Finally, click the **Create Content Type** to create a custom genetic marker content type.  We are provided with a message indicating that a job has been added for creation of the content type.  Depending how large the Chado database is, creation of a contnet type may take awhile, hence we must run it as a job.
+
+.. image:: creating_content.create6.png
 
 Once the content type is created, you can create pages for site visitors. This will be described later in this User's Guide. In short, you can manually create new records through brand new web forms that are created automatically for your content type, or you can use a data loader to import your data directly to Chado, then **Publish** those records through the Tripal interface.
+
+.. note::
+
+  Each time you create a new content type, you get several new things:
+
+  - A new search tool will be created automatically for the content type.
+  - A new set of permissions to help you control access is created.
+
+SNP Example
+^^^^^^^^^^^
+Perhaps we want to be more specific with our genetic marker pages and create pages for each type of genetic marker (e.g. SNP, RFLP, etc. pages).  Suppose for this example that we continue to store genetic markes in the feature table and use the genetic_marker term in  **type_id** as in the previous example.  To differentiate between different markers, we store a record in the **featureprop** table where the **featureprop.type_id** indicates the that the property provides the marker type and the **featureprop.value** column houses the string for the marker type (e.g. "SNP").  Thus, any genetic marker that has a property with this type of featureprop should form part of our SNP content type.
+
+To accomplish this we can walk through the content type creation form and set the following values:
+
+.. csv-table::
+  :header: 'Field', 'Value'
+
+  "Content Type", "SNP (SO:0000694)"
+  "Storage Backend", "Chado"
+  "Chado Table", "feature"
+  "Are all records in the 'feature' table of type 'genetic_marker'?", "No"
+  "Type column", "--None--",
+  "Do you want to use the 'featureprop' table to distinguish between content types?", "Yes"
+  "Base Type", "genetic_marker (SO:0001645)"
+  "Property Type", "type (rdfs:type)"
+  "Property Value", "SNP"
+
+.. image:: creating_content.create7.png
+
+After clicking the **Create content type** button a job will be submitted and we will have a new SNP content type whose data is saved to both the feature and featureprop tables.

legacy/tripal_contact/tripal_contact.install

@@ -51,14 +51,6 @@ function tripal_contact_requirements($phase) {
  */
 function tripal_contact_install() {
 
-  // Add the contactprop table to Chado.
-  tripal_contact_add_custom_tables();
-
-  // Add loading of the the tripal contact ontology to the job queue.
-  $obo_path = '{tripal_contact}/files/tcontact.obo';
-  $obo_id = tripal_insert_obo('Tripal Contacts', $obo_path);
-  tripal_submit_obo_job(array('obo_id' => $obo_id));
-
   // Add cvterms for relationship types.
   tripal_contact_add_cvs();
   tripal_contact_add_cvterms();
@@ -75,13 +67,7 @@ function tripal_contact_install() {
  * @ingroup tripal_legacy_contact
  */
 function tripal_contact_uninstall() {
-  /*
-  // remove our custom block visibility settings per node type
-  db_delete('block_node_type')
-    ->condition('module', 'chado_contact')
-    ->condition('delta', 'contbase')
-    ->execute();
-    */
+
 }
 
 /**
@@ -166,71 +152,3 @@ function tripal_contact_schema() {
   return $schema;
 }
 
-/**
- * Add any custom tables needed by this module.
- * - Contactprop: keep track of properties of contact
- *
- * @ingroup tripal_legacy_contact
- */
-// This function was moved to tripal_chado/includes/setup/tripal_chado.setup.inc
-/* function tripal_contact_add_custom_tables(){
-  $schema = array (
-    'table' => 'contactprop',
-    'fields' => array (
-      'contactprop_id' => array (
-        'type' => 'serial',
-        'not null' => true,
-      ),
-      'contact_id' => array (
-        'type' => 'int',
-        'not null' => true,
-      ),
-      'type_id' => array (
-        'type' => 'int',
-        'not null' => true,
-      ),
-      'value' => array (
-        'type' => 'text',
-        'not null' => false,
-      ),
-      'rank' => array (
-        'type' => 'int',
-        'not null' => true,
-        'default' => 0,
-      ),
-    ),
-    'primary key' => array (
-      0 => 'contactprop_id',
-    ),
-    'unique keys' => array (
-      'contactprop_c1' => array (
-        0 => 'contact_id',
-        1 => 'type_id',
-        2 => 'rank',
-      ),
-    ),
-    'indexes' => array (
-      'contactprop_idx1' => array (
-        0 => 'contact_id',
-      ),
-      'contactprop_idx2' => array (
-        0 => 'type_id',
-      ),
-    ),
-    'foreign keys' => array (
-      'cvterm' => array (
-        'table' => 'cvterm',
-        'columns' => array (
-          'type_id' => 'cvterm_id',
-        ),
-      ),
-      'contact' => array (
-        'table' => 'contact',
-        'columns' => array (
-          'contact_id' => 'contact_id',
-        ),
-      ),
-    ),
-  );
-  chado_create_custom_table('contactprop', $schema, TRUE);
-} */

legacy/tripal_pub/api/tripal_pub.DEPRECATED.inc

@@ -129,11 +129,11 @@ function tripal_pub_import_publications_by_import_id($import_id, $job_id = NULL)
     "DEPRECATED: %old_function has been replaced with %new_function. Please update your code.",
     array(
       '%old_function'=>'tripal_pub_import_publications_by_import_id',
-      '%new_function' => 'tripal_execute_pub_importer'
+      '%new_function' => 'chado_execute_pub_importer'
     )
   );
 
-  return tripal_execute_pub_importer($import_id, $job_id);
+  return chado_execute_pub_importer($import_id, TRUE, FALSE, $job_id);
 }
 
 /**

legacy/tripal_pub/tripal_pub.install

@@ -16,7 +16,7 @@ function tripal_pub_disable() {
   require_once("tripal_pub.views_default.inc");
   $views = tripal_pub_views_default_views();
   foreach (array_keys($views) as $view_name) {
-    tripal_disable_view($view_name,FALSE,array('suppress_error' => TRUE));
+    tripal_disable_view($view_name, FALSE);
   }
 
 }
@@ -49,17 +49,6 @@ function tripal_pub_requirements($phase) {
 function tripal_pub_install() {
   global $base_path;
 
-  // add loading of the the tripal pub ontology to the job queue
-  $obo_path = '{tripal_pub}/files/tpub.obo';
-  $obo_id = tripal_insert_obo('Tripal Publication', $obo_path);
-  tripal_submit_obo_job(array('obo_id' => $obo_id));
-
-  tripal_pub_add_cvs();
-  tripal_pub_add_cvterms();
-
-  // add the custom tables
-  tripal_pub_add_custom_tables();
-
   // set the default vocabularies
   tripal_set_default_cv('pub', 'type_id', 'tripal_pub');
   tripal_set_default_cv('pubprop', 'type_id', 'tripal_pub');
@@ -126,95 +115,9 @@ function tripal_pub_schema() {
     'primary key' => array('nid'),
   );
 
-  $schema['tripal_pub_import'] = array(
-    'fields' => array(
-      'pub_import_id' => array(
-        'type' => 'serial',
-        'not null' => TRUE
-      ),
-      'name' => array(
-        'type' => 'varchar',
-        'length' => 255,
-        'not null' => TRUE
-      ),
-      'criteria' => array(
-        'type' => 'text',
-        'size' => 'normal',
-        'not null' => TRUE,
-        'description' => 'Contains a serialized PHP array containing the search criteria'
-      ),
-      'disabled'  => array(
-        'type' => 'int',
-        'unsigned' => TRUE,
-        'not NULL' => TRUE,
-        'default' => 0
-      ),
-      'do_contact'  => array(
-        'type' => 'int',
-        'unsigned' => TRUE,
-        'not NULL' => TRUE,
-        'default' => 0
-      ),
-    ),
-    'primary key' => array('pub_import_id'),
-    'indexes' => array(
-      'name' => array('name')
-    ),
-  );
-
   return $schema;
 }
 
-/**
- * Add custom table related to publications
- *  - pubauthor_contact
- *
- * @ingroup tripal_legacy_pub
- */
-// This function was moved to tripal_chado/includes/setup/tripal_chado.setup.inc
-/* function tripal_pub_add_custom_tables() {
-  $schema = array (
-    'table' => 'pubauthor_contact',
-    'fields' => array (
-      'pubauthor_contact_id' => array (
-        'type' => 'serial',
-        'not null' => true,
-      ),
-      'contact_id' => array (
-        'type' => 'int',
-        'not null' => true,
-      ),
-      'pubauthor_id' => array (
-        'type' => 'int',
-        'not null' => true,
-      ),
-    ),
-    'primary key' => array (
-      0 => 'pubauthor_contact_id',
-    ),
-    'unique keys' => array (
-      'pubauthor_contact_c1' => array (
-        0 => 'contact_id',
-        1 => 'pubauthor_id',
-      ),
-    ),
-    'foreign keys' => array (
-      'contact' => array (
-        'table' => 'contact',
-        'columns' => array (
-          'contact_id' => 'contact_id',
-        ),
-      ),
-      'pubauthor' => array (
-        'table' => 'pubauthor',
-        'columns' => array (
-          'pubauthor_id' => 'pubauthor_id',
-        ),
-      ),
-    ),
-  );
-  chado_create_custom_table('pubauthor_contact', $schema, TRUE);
-} */
 
 /**
  * This is the required update for tripal_pub when upgrading from Drupal core API 6.x.

legacy/tripal_pub/tripal_pub.module

@@ -265,31 +265,6 @@ function tripal_pub_permission() {
   );
 }
 
-/**
- * Implements hook_mail().
- *
- * @ingroup tripal_legacy_pub
- */
-function tripal_pub_mail($key, &$message, $params) {
-  $site_name = variable_get('site_name');
-  $language = $message['language'];
-  switch($key) {
-    case 'import_report':
-      $headers = array(
-        'MIME-Version' => '1.0',
-        'Content-Type' => 'text/html; charset=UTF-8; format=flowed',
-        'Content-Transfer-Encoding' => '8Bit',
-        'X-Mailer' => 'Drupal'
-      );
-      foreach ($headers as $key => $value) {
-        $message['headers'][$key] = $value;
-      }
-      $message['subject'] = t('Publication import from !site', array('!site' => $site_name));
-      $message['body'][] = $params['message'];
-      break;
-  }
-}
-
 /**
  * Implementation of hook_form_alter().
  *
@@ -338,7 +313,7 @@ function tripal_pub_form_alter(&$form, &$form_state, $form_id) {
 function tripal_pub_job_describe_args($callback, $args) {
 
   $new_args = array();
-  if ($callback == 'tripal_execute_pub_importer') {
+  if ($callback == 'chado_execute_pub_importer') {
     // get all of the loaders
     $qargs = array(':import_id' => $args[0]);
     $sql = "SELECT * FROM {tripal_pub_import} WHERE pub_import_id = :import_id ";

tripal/api/tripal.entities.api.inc

@@ -380,15 +380,14 @@ function tripal_load_bundle_entity($values) {
  */
 function tripal_add_notification($title, $details, $type, $actions, $submitter_id) {
   $transaction = db_transaction();
-
-  // Check the notification isn't already in the admin notification table.
-  $dedup = db_select('tripal_admin_notfications', 'tan')
-    ->fields('tan')
-    ->condition('submitter_id', $submitter_id, '=')
-    ->execute()->fetchAll();
-
-  if (empty($dedup)) {
-    try {
+  try {
+    // Check the notification isn't already in the admin notification table.
+    $dedup = db_select('tripal_admin_notfications', 'tan')
+      ->fields('tan')
+      ->condition('submitter_id', $submitter_id, '=')
+      ->execute()->fetchAll();
+  
+    if (empty($dedup)) {
       $record = new stdClass;
       $record->details = $details;
       $record->title = $title;
@@ -398,10 +397,10 @@ function tripal_add_notification($title, $details, $type, $actions, $submitter_i
       $record->type = $type;
       $success = drupal_write_record('tripal_admin_notfications', $record);
     }
-    catch (Exception $e) {
-      $transaction->rollback();
-      watchdog('tripal_cron', 'Could not write notification to database.');
-    }
+  }
+  catch (Exception $e) {
+    $transaction->rollback();
+    watchdog('tripal_cron', 'Could not write notification to database.');
   }
 }
 

tripal/api/tripal.importer.api.inc

@@ -210,10 +210,9 @@ function tripal_run_importer($import_id, TripalJob $job = NULL) {
  */
 function tripal_run_importer_run($loader, $job) {
 
+  // begin the transaction
+  $transaction = db_transaction();
   try {
-    // begin the transaction
-    $transaction = db_transaction();
-
     $loader->run();
 
     if ($job) {
@@ -244,9 +243,9 @@ function tripal_run_importer_run($loader, $job) {
  * @ingroup tripal_importer_api
  */
 function tripal_run_importer_post_run($loader, $job) {
-  try {
-    // the transaction
-    $transaction = db_transaction();
+  // the transaction
+  $transaction = db_transaction();
+  try {    
     $loader->postRun();
   }
   catch (Exception $e) {

tripal/api/tripal.jobs.api.inc

@@ -567,4 +567,4 @@ function tripal_execute_job($job_id, $redirect = TRUE) {
   if ($redirect) {
     drupal_goto("admin/tripal/tripal_jobs/view/$job_id");
   }
-}
+}

tripal/api/tripal.notice.api.inc

@@ -31,8 +31,13 @@ define('TRIPAL_DEBUG',7);
 /**
  * Provide better error notice for Tripal.
  *
+ * Please be sure to set the $options array as desired. For example, by default
+ * this function sends all messages to the Drupal watchdog. If a long running
+ * job uses this function and prints status messages you may not want to have
+ * those go to the watchdog as it can dramatically affect performance. 
+ * 
  * If the environment variable 'TRIPAL_DEBUG' is set to 1 then this function
- * will add backtrace information to the message.
+ * will add backtrace information to the message.  
  *
  * @param $type
  *   The catagory to which this message belongs. Can be any string, but the
@@ -60,6 +65,12 @@ define('TRIPAL_DEBUG',7);
  *       display is the command-line
  *     - drupal_set_message: set to TRUE to call drupal_set_message with the 
  *       same error.
+ *     - drupal_set_message:  set to TRUE then send the message to the
+ *       drupal_set_message function.
+ *     - watchdog:  set to FALSE to disable logging to Drupal's watchdog.
+ *     - job: The jobs management object for the job if this function is run 
+ *       as a job. Adding the job object here ensures that any status or error
+ *       messages are also logged with the job.
  *
  * @ingroup tripal_notify_api
  */
@@ -110,13 +121,15 @@ function tripal_report_error($type, $severity, $message, $variables = array(), $
     }
   }
 
-  // Send to watchdog.
-  try {
-    watchdog($type, $message, $variables, $severity);
-  }
-  catch (Exception $e) {
-    print "CRITICAL (TRIPAL): Unable to add error message with watchdog: " . $e->getMessage(). "\n.";
-    $options['print'] = TRUE;
+  // Send to watchdog if the user wants.
+  if (array_key_exists('watchdog', $options) and $options['watchdog'] !== FALSE) {   
+    try {
+      watchdog($type, $message, $variables, $severity);
+    }
+    catch (Exception $e) {
+      print "CRITICAL (TRIPAL): Unable to add error message with watchdog: " . $e->getMessage(). "\n.";
+      $options['print'] = TRUE;
+    }
   }
 
   // Format the message for printing (either to the screen, log or both).
@@ -149,7 +162,10 @@ function tripal_report_error($type, $severity, $message, $variables = array(), $
   if (($severity != TRIPAL_INFO)) {
     tripal_log('[' . strtoupper($type) . '] ' . $print_message . "\n", $severity_string);
   }
-
+  
+  if (array_key_exists('job', $options) and is_a($options['job'], 'TripalJob')) {   
+    $options['job']->logMessage($message, $variables, $severity);
+  }
 }
 
 /**
@@ -231,8 +247,12 @@ function tripal_set_message($message, $importance = TRIPAL_INFO, $options = arra
 }
 
 /**
- * File-based logging for Tripal.
- *
+ * File-based error logging for Tripal.
+ * 
+ * Consider using the tripal_report_error function rather than
+ * calling this function directly, as that function calls this one for non
+ * INFO messages and has greater functionality. 
+ * 
  * @param $message
  *   The message to be logged. Need not contain date/time information.
  * @param $log_type
@@ -240,7 +260,7 @@ function tripal_set_message($message, $importance = TRIPAL_INFO, $options = arra
  *   are supported.
  * @param $options
  *   An array of options where the following keys are supported:
- *     - first_progress_bar: this sohuld be used for the first log call for a
+ *     - first_progress_bar: this should be used for the first log call for a
  *       progress bar.
  *     - is_progress_bar: this option should be used for all but the first print
  *       of a progress bar to allow it all to be printed on the same line

tripal/api/tripal.terms.api.inc

@@ -61,43 +61,6 @@ function hook_vocab_storage_info() {
   );
 }
 
-
-/**
- * Creates a form for specifying a term for TripalEntity creation.
- *
- * This hook allows the module that implements a vocabulary storage backend
- * to provide the form necessary to select a term that will then be used for
- * creating a new TripalEntity type.  Tripal will expect that a 'vocabulary' and
- * 'accession' are in the $form_state['storage'] array. The 'vocabulary' and
- * must be the abbreviated uppercase vocabulary for the vocabulary (e.g. 'RO',
- * 'SO', 'PATO', etc.).  The 'accession' must be the unique term ID (or
- * accession) for the term in the vocabulary.
- *
- * @param $form
- * @param $form_state
- *
- * @return
- *   A form object.
- *
- * @ingroup tripal_terms_api
- */
-function hook_vocab_select_term_form(&$form, &$form_state) {
-
-  return $form;
-}
-
-/**
- * Validates the hook_vocab_select_term_form().
- *
- * @param $form
- * @param $form_state
- *
- * @ingroup tripal_terms_api
- */
-function hook_vocab_select_term_form_validate($form, &$form_state) {
-
-}
-
 /**
  * Provides a form for importing vocabularies and their terms.
  *
@@ -618,19 +581,32 @@ function tripal_get_vocabularies() {
  */
 function tripal_get_term_lookup_form(&$form, &$form_state, $default_name = '',
     $title = 'Vocabulary Term', $description = '', $is_required,
-    $field_name = '', $delta = 0 ) {
+    $field_name = '', $delta = 0, $callback = '', $wrapper = '', $validate = [], 
+    $weight = 0) {
   
-  $ajax_wrapper_id = 'tripal-vocab-select-form-' . $delta;
-  if ($field_name) {
-    $ajax_wrapper_id = $field_name . '-' . $delta;
+  if (!$callback) {
+    $callback = 'tripal_get_term_lookup_form_ajax_callback';
   }
-
+  
+  if (!$wrapper) {
+    $ajax_wrapper_id = 'tripal-vocab-select-form-' . $delta;
+    if ($field_name) {
+      $ajax_wrapper_id = $field_name . '-' . $delta;
+    }
+  }
+  else {
+    $ajax_wrapper_id = $wrapper;
+  }
+  
   $term_name = $default_name;
-  if (array_key_exists('values', $form_state) and array_key_exists('term_name', $form_state['values'])) {
-    $term_name = $form_state['values']['term_name'];
+  if (array_key_exists('values', $form_state) and array_key_exists('term_name' . $delta, $form_state['values'])) {
+    $term_name = $form_state['values']['term_name' . $delta];
+  }
+  if (array_key_exists('input', $form_state) and array_key_exists('term_name' . $delta, $form_state['input'])) {
+    $term_name = $form_state['input']['term_name' . $delta];
   }
   if ($field_name and array_key_exists('input', $form_state) and array_key_exists($field_name, $form_state['input'])) {
-    $term_name = $form_state['input'][$field_name]['und'][$delta]['term_match']['term_name'];
+    $term_name = $form_state['input'][$field_name]['und'][$delta]['term_match' . $delta]['term_name' . $delta];
   }
 
   if (!$description) {
@@ -642,15 +618,16 @@ function tripal_get_term_lookup_form(&$form, &$form_state, $default_name = '',
   $form_state['storage'][$ajax_wrapper_id]['term_match_field'] = $field_name;
   $form_state['storage'][$ajax_wrapper_id]['term_match_delta'] = $delta;
 
-  $form['term_match'] = array(
+  $form['term_match' . $delta] = array(
     '#type' => 'fieldset',
     '#collapsible' => FALSE,
     '#collapsed' => FALSE,
     '#title' => t($title),
     '#prefix' => '<div id = "' . $ajax_wrapper_id . '">',
     '#suffix' => '</div>',
+    '#weight' => $weight,
   );
-  $form['term_match']['term_name'] = array(
+  $form['term_match' . $delta]['term_name' . $delta] = array(
     '#title'  => t('Type'),
     '#type' => 'textfield',
     '#description' => $description,
@@ -658,30 +635,33 @@ function tripal_get_term_lookup_form(&$form, &$form_state, $default_name = '',
     '#default_value' => $term_name,
     '#autocomplete_path' => "admin/tripal/storage/chado/auto_name/cvterm/",
   );
-  $form['term_match']['select_button'] = array(
+  $form['term_match' . $delta]['select_button' . $delta] = array(
     '#type' => 'button',
     '#value' => t('Lookup Term'),
     '#name' => 'select_cvterm_' . $ajax_wrapper_id,
-    '#validate' => array(),
-    '#limit_validation_errors' => array(),
+    '#validate' => $validate,
     '#ajax' => array(
-      'callback' => "tripal_get_term_lookup_form_ajax_callback",
+      'callback' => $callback,
       'wrapper' => $ajax_wrapper_id,
       'effect' => 'fade',
       'method' => 'replace'
     ),
   );
+  if (empty($validate)) {
+    $form['term_match' . $delta]['select_button' . $delta]['#limit_validation_errors'] = [];
+  }
+  
 
   // If the term has been provided by the user then we want to search for
   // matching terms in the database and let them select among any matches.
   if ($term_name) {
     $submit_disabled = TRUE;
-    $form['term_match']['terms_list'] = array(
+    $form['term_match' . $delta]['terms_list' . $delta] = array(
       '#type' => 'fieldset',
       '#title' => t('Matching Terms'),
-      '#description' => t('Please select the term the best matches the
-          content type you want to create. If the same term exists in
-          multiple vocabularies you will see more than one option below.')
+      '#description' => t('Please select the best matching term. If the 
+        same term exists in multiple vocabularies you will see more than 
+        one option below.')
     );
     $match = array(
       'name' => $term_name,
@@ -700,27 +680,22 @@ function tripal_get_term_lookup_form(&$form, &$form_state, $default_name = '',
     foreach ($terms as $term) {
       // Save the user a click by setting the default value as 1 if there's
       // only one matching term.
-      $default = FALSE;
+      $checked = FALSE;
       $attrs = array();
       if ($num_terms == 0 and count($terms) == 1) {
-        $default = TRUE;
+        $checked = TRUE;
         $attrs = array('checked' => 'checked');
       }
-      $term_element_name = 'term-' . $term->cvterm_id;
-      $form['term_match']['terms_list'][$term_element_name] = array(
+      $term_element_name = 'term-' . $term->cvterm_id  . '-' . $delta;
+      $definition = property_exists($term, 'definition') ? $term->definition : '';
+      $form['term_match' . $delta]['terms_list' . $delta][$term_element_name] = array(
         '#type' => 'checkbox',
         '#title' =>  $term->name,
-        '#default_value' => $default,
+        '#default_value' => $checked,
         '#attributes' => $attrs,
         '#description' => '<b>Vocabulary:</b> ' . $term->cv_id->name . ' (' . $term->dbxref_id->db_id->name . ') ' . $term->cv_id->definition .
-        '<br><b>Term: </b> ' . $term->dbxref_id->db_id->name . ':' . $term->dbxref_id->accession . '.  ' .
-        '<br><b>Definition:</b>  ' . property_exists($term, 'definition') ? $term->definition : '',
-        '#ajax' => array(
-          'callback' => "tripal_get_term_lookup_form_ajax_callback",
-          'wrapper' => $ajax_wrapper_id,
-          'effect' => 'fade',
-          'method' => 'replace'
-        ),
+          '<br><b>Term ID: </b> ' . $term->dbxref_id->db_id->name . ':' . $term->dbxref_id->accession . '.  ' .
+          '<br><b>Definition:</b>  ' . $definition,
       );
 
       if (array_key_exists('values', $form_state) and array_key_exists($term_element_name, $form_state['values']) and
@@ -741,40 +716,35 @@ function tripal_get_term_lookup_form(&$form, &$form_state, $default_name = '',
       $term = $synonym->cvterm_id;
       // Save the user a click by setting the default value as 1 if there's
       // only one matching term.
-      $default = FALSE;
+      $checked = FALSE;
       $attrs = array();
       if ($num_terms == 0 and count($terms) == 1) {
-        $default = TRUE;
+        $checked = TRUE;
         $attrs = array('checked' => 'checked');
       }
-      $term_element_name = 'term-' . $term->cvterm_id;
-      $form['term_match']['terms_list'][$term_element_name] = array(
+      $term_element_name = 'term-' . $term->cvterm_id . '-' . $delta;
+      $definition = property_exists($term, 'definition') ? $term->definition : '';
+      $form['term_match' . $delta]['terms_list' . $delta][$term_element_name] = array(
         '#type' => 'checkbox',
         '#title' =>  $term->name,
-        '#default_value' => $default,
+        '#default_value' => $checked,
         '#attributes' => $attrs,
         '#description' => '<b>Vocabulary:</b> ' . $term->cv_id->name . ' (' . $term->dbxref_id->db_id->name . ') ' . $term->cv_id->definition .
         '<br><b>Term: </b> ' . $term->dbxref_id->db_id->name . ':' . $term->dbxref_id->accession . '.  ' .
-        '<br><b>Definition:</b>  ' . $term->definition . 
+        '<br><b>Definition:</b>  ' . $definition . 
         '<br><b>Synonym:</b> ' . $synonym->synonym,
-        '#ajax' => array(
-          'callback' => "tripal_get_term_lookup_form_ajax_callback",
-          'wrapper' => $ajax_wrapper_id,
-          'effect' => 'fade',
-          'method' => 'replace'
-        ),
       );
       
       if (array_key_exists('values', $form_state) and array_key_exists($term_element_name, $form_state['values']) and
         $form_state['values'][$term_element_name] == 1) {
-          $selected_term = $term;
-        }
-        $num_terms++;
+        $selected_term = $term;
+      }
+      $num_terms++;
     }
     
     
     if ($num_terms == 0) {
-      $form['term_match']['terms_list']['none'] = array(
+      $form['term_match'. $delta]['terms_list'. $delta]['none'. $delta] = array(
         '#type' => 'item',
         '#markup' => '<i>' . t('There is no term that matches the entered text.') . '</i>'
       );
@@ -804,19 +774,19 @@ function tripal_get_term_lookup_form_result($form, $form_state, $field_name = ''
   $values = array();
   $selected = array();
   if ($field_name) {
-    if (array_key_exists('term_match', $form_state['values'][$field_name]['und'][$delta]) and 
-      array_key_exists('terms_list', $form_state['values'][$field_name]['und'][$delta]['term_match'])) {
-      $values = $form_state['values'][$field_name]['und'][$delta]['term_match']['terms_list'];
+    if (array_key_exists('term_match' . $delta, $form_state['values'][$field_name]['und'][$delta]) and 
+        array_key_exists('terms_list' . $delta, $form_state['values'][$field_name]['und'][$delta]['term_match'. $delta])) {
+      $values = $form_state['values'][$field_name]['und'][$delta]['term_match'. $delta]['terms_list'. $delta];
     }
   }
   else {
-    $values = $form_state['values'];
+    $values = array_key_exists('values', $form_state) ? $form_state['values'] : [];
   }
 
   if (is_array($values)) {
     foreach ($values as $key => $value) {
       $matches = array();
-      if (preg_match("/^term-(\d+)$/", $key, $matches) and $values['term-' . $matches[1]]) {
+      if (preg_match("/^term-(\d+)-$delta$/", $key, $matches) and $values['term-' . $matches[1] . '-' . $delta]) {
         $cvterm_id = $matches[1];
         $selected[] = chado_generate_var('cvterm', array('cvterm_id' => $cvterm_id));
       }
@@ -839,9 +809,9 @@ function tripal_get_term_lookup_form_ajax_callback($form, $form_state) {
   // If this form is in a field then we need to dig a bit deeper to return
   // the form elements.
   if ($field_name) {
-    return $form[$field_name]['und'][$delta]['term_match'];
+    return $form[$field_name]['und'][$delta]['term_match'. $delta];
   }
   else {
-    return $form['term_match'];
+    return $form['term_match' . $delta];
   }
 }

tripal/includes/TripalBundleController.inc

@@ -51,59 +51,70 @@ class TripalBundleController extends EntityAPIControllerExportable {
     if (!$transaction) {
       $transaction = db_transaction();
     }
-
-    if ($bundles) {
-
-      foreach ($bundles as $id => $bundle) {
-
-        // Allow modules to perform actions when the bundle is deleted.
-        $modules = module_implements('bundle_delete');
-        foreach ($modules as $module) {
-          $function = $module . '_bundle_delete';
-          $function($bundle);
+    
+    try {
+
+      if ($bundles) {
+  
+        foreach ($bundles as $id => $bundle) {
+  
+          // Allow modules to perform actions when the bundle is deleted.
+          $modules = module_implements('bundle_delete');
+          foreach ($modules as $module) {
+            $function = $module . '_bundle_delete';
+            $function($bundle);
+          }
+  
+          // Find any TripalEntity fields that are attached to this bundle and
+          // remove them.
+          $instances = field_info_instances('TripalEntity', $bundle->name);
+          foreach ($instances as $instance) {
+            // Mark the instance as deleted and purge it.
+            $field = field_info_field($instance['field_name']);
+            field_delete_instance($instance);
+            field_purge_instance($instance);
+  
+            // If the field has no more instances then purge it too.
+            if (count($field['bundles']) == 1 and
+              count($field['bundles']['TripalEntity']) == 1 and
+              in_array($bundle->name, $field['bundles']['TripalEntity'])
+            ) {
+              field_purge_field($field);
+            }
+          }
+  
+          // Remove any entities from the tripal_entity table.
+          db_delete('tripal_entity')
+            ->condition('bundle', $bundle->name)
+            ->execute();
+  
+          // Remove the terms for the bundles that are to be deleted.
+          db_delete('tripal_term')
+            ->condition('id', $bundle->term_id)
+            ->execute();
         }
-
-        // Find any TripalEntity fields that are attached to this bundle and
-        // remove them.
-        $instances = field_info_instances('TripalEntity', $bundle->name);
-        foreach ($instances as $instance) {
-          // Mark the instance as deleted and purge it.
-          $field = field_info_field($instance['field_name']);
-          field_delete_instance($instance);
-          field_purge_instance($instance);
-
-          // If the field has no more instances then purge it too.
-          if (count($field['bundles']) == 1 and
-            count($field['bundles']['TripalEntity']) == 1 and
-            in_array($bundle->name, $field['bundles']['TripalEntity'])
-          ) {
-            field_purge_field($field);
+  
+        // Use the parent function to delete the bundles.
+        parent::delete($ids, $transaction);
+  
+        // Not sure what this does, but copied from the
+        // EntityAPIControllerExportable->delete() function which this one
+        // overrides.
+        foreach ($bundles as $id => $bundle) {
+          if (entity_has_status($this->entityType, $bundle, ENTITY_IN_CODE)) {
+            entity_defaults_rebuild([$this->entityType]);
+            break;
           }
         }
-
-        // Remove any entities from the tripal_entity table.
-        db_delete('tripal_entity')
-          ->condition('bundle', $bundle->name)
-          ->execute();
-
-        // Remove the terms for the bundles that are to be deleted.
-        db_delete('tripal_term')
-          ->condition('id', $bundle->term_id)
-          ->execute();
       }
-
-      // Use the parent function to delete the bundles.
-      parent::delete($ids, $transaction);
-
-      // Not sure what this does, but copied from the
-      // EntityAPIControllerExportable->delete() function which this one
-      // overrides.
-      foreach ($bundles as $id => $bundle) {
-        if (entity_has_status($this->entityType, $bundle, ENTITY_IN_CODE)) {
-          entity_defaults_rebuild([$this->entityType]);
-          break;
-        }
+    }
+    catch (Exception $e) {
+      if ($transaction) {
+        $transaction->rollback();
       }
+      watchdog_exception('tripal', $e);
+      throw $e;
+      return FALSE;
     }
   }
 }

tripal/includes/TripalBundleUIController.inc

@@ -537,6 +537,7 @@ function tripal_bundle_access($op, $type = NULL, $account = NULL) {
  */
 function tripal_admin_add_type_form($form, &$form_state) {
 
+  // Make sure we have a storage backend for managing content types.
   $stores = module_invoke_all('vocab_storage_info');
   if (!is_array($stores) or count($stores) == 0) {
     tripal_set_message('A storage backend is not enabled for managing
@@ -545,207 +546,218 @@ function tripal_admin_add_type_form($form, &$form_state) {
           and return to create new Tripal content types.', TRIPAL_NOTICE);
     return;
   }
-  $keys = array_keys($stores);
-  $module = $stores[$keys[0]]['module'];
-  $function = $module . '_vocab_select_term_form';
-  if (function_exists($function)) {
-    $form = $function($form, $form_state);
+  
+  // Set the stage to step1 if it isn't already set.
+  if (!isset($form_state['stage'])) $form_state['stage'] = 'step1';
+  $stage = $form_state['stage'];
+  
+  
+  // Get the selected term.
+  if (array_key_exists('values', $form_state) and
+      array_key_exists('term', $form_state['values'])) {
+    $selected_term = $form_state['values']['term'];
+  }
+  else {
+    $selected = tripal_get_term_lookup_form_result($form, $form_state);
+    if ($selected) {
+      $selected_term = $selected[0];
+    }
+  }
+    
+  // Get the selected storage element.
+  $default_store = 'term_chado_storage';
+  if (array_key_exists('values', $form_state) and
+      array_key_exists('store_select', $form_state['values'])) {
+    $default_store = $form_state['values']['store_select'];
   }
+  
 
-  $term_name = array_key_exists('values', $form_state) ? $form_state['values']['term_name'] : '';
+  // Handle the different stages:
+  if ($stage == 'step1') {
+    tripal_admin_add_type_form_step1($form, $form_state);
+  }
+  if ($form_state['stage'] == 'step2') {
+    tripal_admin_add_type_form_step1_summary($form, $form_state, $selected_term);
+    tripal_admin_add_type_form_step2($form, $form_state, $stores, $selected_term, $default_store);
+  }
+  if ($form_state['stage'] == 'step3') {
+    tripal_admin_add_type_form_step1_summary($form, $form_state, $selected_term);
+    tripal_admin_add_type_form_step2_summary($form, $form_state, $stores, $selected_term, $default_store);
+    tripal_admin_add_type_form_step3($form, $form_state, $stores, $selected_term, $default_store);
+  }
+  
+  $form['#prefix'] = '<div id = "tripal-add-type-form">';
+  $form['#suffix'] = '</div>';
+  
+  return $form;
+}
 
-  // If no term has been selected yet then provide the auto complete field.
-  $form['term_name'] = array(
-    '#title'       => t('Content Type'),
-    '#type'        => 'textfield',
-    '#description' => t("The content type must be the name of a term in
+/**
+ * Builds step1 of the tripal_admin_add_type_form()
+ */
+function tripal_admin_add_type_form_step1(&$form, &$form_state) {
+  
+  // Get the term name from the form_state.
+  $term_name = '';
+  if (array_key_exists('values', $form_state) and array_key_exists('term_name0', $form_state['values'])) {
+    $term_name = $form_state['values']['term_name0'];
+  }
+  if (array_key_exists('input', $form_state) and array_key_exists('term_name0', $form_state['input'])) {
+    $term_name = $form_state['input']['term_name0'];
+  }
+  
+  // Get the term lookup form.
+  $description = t("The content type must be the name of a term in
         a controlled vocabulary and the controlled vocabulary should
         already be loaded into Tripal.  For example, to create a content
         type for storing 'genes', use the 'gene' term from the
-        Sequence Ontology (SO)."),
-    '#required'    => TRUE,
-    '#default_value' => $term_name,
-    '#autocomplete_path' => "admin/tripal/storage/chado/auto_name/cvterm/",
-  );
-  $form['select_button'] = array(
-    '#type' => 'submit',
-    '#value' => t('Lookup Term'),
-    '#name' => 'select_cvterm',
-    '#ajax' => array(
-      'callback' => "tripal_admin_add_type_form_ajax_callback",
-      'wrapper' => "tripal-vocab-select-form",
-      'effect' => 'fade',
-      'method' => 'replace'
-    ),
-  );
-  $form['#prefix'] = '<div id = "tripal-vocab-select-form">';
-  $form['#suffix'] = '</div>';
-
-  // If the term has been provided by the user then we want to search for
-  // matching terms in the database and let them select among any matches.
+        Sequence Ontology (SO).");
+  tripal_get_term_lookup_form($form, $form_state, $term_name,
+    'Step 1: Content Type', $description, TRUE, '', 0,
+    'tripal_admin_add_type_form_ajax_callback');
+  
   if ($term_name) {
-    $submit_disabled = TRUE;
-    $form['terms_list'] = array(
-      '#type' => 'fieldset',
-      '#title' => t('Matching Terms'),
-      '#description' => t('Please select the term the best matches the
-          content type you want to create. If the same term exists in
-          multiple vocabularies you will see more than one option below.')
-    );
-    $match = array(
-      'name' => $term_name,
+    $form['term_match']['step1-continue'] = array(
+      '#type' => 'submit',
+      '#value' => t('Continue'),
+      '#name' => 'step1-continue',
     );
-    $terms = chado_generate_var('cvterm', $match, array('return_array' => TRUE));
-    $terms = chado_expand_var($terms, 'field', 'cvterm.definition');
-    $num_terms = 0;
-    $selected_term = '';
-
-    // Let the user select from any matching terms. Sometimes there may be
-    // more than one that match.
-    foreach ($terms as $term) {
-      // Save the user a click by setting the default value as 1 if there's
-      // only one matching term. 
-      $default = FALSE;
-      $attrs = array();
-      if ($num_terms == 0 and count($terms) == 1) {
-        $default = TRUE;
-        $attrs = array('checked' => 'checked');
-      }
-      $term_element_name = 'term-' . $term->cvterm_id;
-      $form['terms_list'][$term_element_name] = array(
-        '#type' => 'checkbox',
-        '#title' =>  $term->name,
-        '#default_value' => $default,
-        '#attributes' => $attrs,
-        '#description' => '<b>Vocabulary:</b> ' . $term->cv_id->name . ' (' . $term->dbxref_id->db_id->name . ') ' . $term->cv_id->definition .
-        '<br><b>Term: </b> ' . $term->dbxref_id->db_id->name . ':' . $term->dbxref_id->accession . '.  ' .
-        '<br><b>Definition:</b>  ' . $term->definition,
-        '#ajax' => array(
-          'callback' => "tripal_admin_add_type_form_ajax_callback",
-          'wrapper' => "tripal-vocab-select-form",
-          'effect' => 'fade',
-          'method' => 'replace'
-        ),
-      );
-
-      if (array_key_exists('values', $form_state) and array_key_exists($term_element_name, $form_state['values']) and
-          $form_state['values'][$term_element_name] == 1) {
-        $selected_term = $term;
-      }
-      $num_terms++;
-    }
-    
-    
-    // Next find terms that are synonyms
-    $match = array(
-      'synonym' => $term_name,
-    );
-    $termsyn = chado_generate_var('cvtermsynonym', $match, array('return_array' => TRUE));
-    // Let the user select from any matching terms. Sometimes there may be
-    // more than one that match.
-    foreach ($termsyn as $synonym) {
-      $term = $synonym->cvterm_id;
-      // Save the user a click by setting the default value as 1 if there's
-      // only one matching term.
-      $default = FALSE;
-      $attrs = array();
-      if ($num_terms == 0 and count($terms) == 1) {
-        $default = TRUE;
-        $attrs = array('checked' => 'checked');
-      }
-      $term_element_name = 'term-' . $term->cvterm_id;
-      $form['term_match']['terms_list'][$term_element_name] = array(
-        '#type' => 'checkbox',
-        '#title' =>  $term->name,
-        '#default_value' => $default,
-        '#attributes' => $attrs,
-        '#description' => '<b>Vocabulary:</b> ' . $term->cv_id->name . ' (' . $term->dbxref_id->db_id->name . ') ' . $term->cv_id->definition .
-        '<br><b>Term: </b> ' . $term->dbxref_id->db_id->name . ':' . $term->dbxref_id->accession . '.  ' .
-        '<br><b>Definition:</b>  ' . $term->definition .
-        '<br><b>Synonym:</b> ' . $synonym->synonym,
-        '#ajax' => array(
-          'callback' => "tripal_admin_add_type_form_ajax_callback",
-          'wrapper' => "tripal-vocab-select-form",
-          'effect' => 'fade',
-          'method' => 'replace'
-        ),
-      );
-      
-      if (array_key_exists('values', $form_state) and array_key_exists($term_element_name, $form_state['values']) and
-        $form_state['values'][$term_element_name] == 1) {
-          $selected_term = $term;
-        }
-        $num_terms++;
-    }
-    if ($num_terms == 0) {
-      $form['terms_list']['none'] = array(
-        '#type' => 'item',
-        '#markup' => '<i>' . t('There is no term that matches the entered text.') . '</i>'
-      );
-      return $form;
-    }
+  }
+}
 
-    // Now let the user select where the data type will be stored.
-    $form['storage'] = array(
-      '#type' => 'fieldset',
-      '#title' => t('Storage Settings'),
-      '#description' => t('The primary record for each content of this type
+/**
+ * Provides a summary of values selected in Step 1. 
+ */
+function tripal_admin_add_type_form_step1_summary(&$form, &$form_state, $selected_term) {
+     
+  $form['term'] = [
+    '#type' => 'value',
+    '#value' => $selected_term,
+  ];
+  
+  $form['term_summary'] = [
+    '#type' => 'fieldset',
+    '#title' => t('Step 1: Content Type'),
+    '#description' => '',
+    '#collapsible' => TRUE,
+    '#collapsed' => TRUE,
+  ];
+  $definition = property_exists($selected_term, 'definition') ? $selected_term->definition : '';
+  $form['term_summary']['details'] = [
+    '#type' => 'item',
+    '#title' => t('Term'),
+    '#markup' => 'Name: ' . $selected_term->name . 
+      '<br>Vocabulary: ' . $selected_term->cv_id->name . ' (' . $selected_term->dbxref_id->db_id->name . ') ' .
+      '<br>Term ID: ' . $selected_term->dbxref_id->db_id->name . ':' . $selected_term->dbxref_id->accession . '.  ' .
+      '<br>Definition:  ' . $definition
+  ];
+  $form['term_summary']['step1-return'] = array(
+    '#type' => 'submit',
+    '#value' => t('Pick a different term'),
+    '#name' => 'step1-return',
+  );  
+}
+  
+/**
+ * Builds step1 of the tripal_admin_add_type_form()
+ */
+function tripal_admin_add_type_form_step2(&$form, &$form_state, $stores, $selected_term, $default_store) {
+  
+  // Now let the user select where the data type will be stored.
+  $form['storage'] = array(
+    '#type' => 'fieldset',
+    '#title' => t('Step 2: Storage'),
+    '#description' => t('The primary record for each content of this type
           must be stored in a single storage backend. Please select the
           storage method and settings for this content type.')
-    );
+  );
+    
+  $store_options = array(0 => '-- Select --');
+  foreach ($stores as $store_type => $store) {
+    $store_options[$store_type] = $store['label'];
+  }
 
-    // TODO: there should be a way for each storage backend to determine if
-    // it can handle the content type.  Maybe certain content types aren't
-    // yet supported by every storage backend.
-    $default_store = 'term_chado_storage';
-    $store_options = array(0 => '-- Select --');
-    foreach ($stores as $store_type => $store) {
-      $store_options[$store_type] = $store['label'];
-    }
-    if (array_key_exists('values', $form_state) and
-        array_key_exists('store_select', $form_state['values'])) {
-      $default_store = $form_state['values']['store_select'];
-    }
-    $form['storage']['store_select'] = array(
-      '#type' => 'select',
-      '#title' => 'Storage backend',
-      '#options' => $store_options,
-      '#default_value' => $default_store,
-      '#ajax' => array(
-        'callback' => "tripal_admin_add_type_form_ajax_callback",
-        'wrapper' => "tripal-vocab-select-form",
-        'effect' => 'fade',
-        'method' => 'replace'
-      ),
-      '#description' => 'Select a storage background for this content type.'
+  $form['storage']['store_select'] = array(
+    '#type' => 'select',
+    '#title' => 'Storage backend',
+    '#options' => $store_options,
+    '#default_value' => $default_store,
+    '#description' => 'Select a storage background for this content type.'
+  );
+  
+  if ($default_store) {
+    $form['term_match']['step2-continue'] = array(
+      '#type' => 'submit',
+      '#value' => t('Continue'),
+      '#name' => 'step2-continue',
     );
+  }
+}
 
-    if ($default_store) {
-      $selected_store_module = $stores[$store_type]['module'];
+/**
+ * Provides a summary of values selected in Step 1.
+ */
+function tripal_admin_add_type_form_step2_summary(&$form, &$form_state, $stores, $selected_term, $default_store) {
+  $default_store = $form_state['values']['store_select'];
+  $selected_store_module = $stores[$default_store]['module'];
+  
+  $form['store_select'] = [
+    '#type' => 'value',
+    '#value' => $default_store,
+  ];
+  
+  $form['store_summary'] = [
+    '#type' => 'fieldset',
+    '#title' => t('Step 2: Storage'),
+    '#description' => '',
+    '#collapsible' => TRUE,
+    '#collapsed' => TRUE,
+  ];
+  $form['store_summary']['details'] = [
+    '#type' => 'item',
+    '#title' => t('Storage backend'),
+    '#markup' => $stores[$default_store]['label']
+  ];
+  $form['term_summary']['step1-return'] = array(
+    '#type' => 'submit',
+    '#value' => t('Pick a different term'),
+    '#name' => 'step1-return',
+  );  
+}
 
-      $function = $selected_store_module . '_field_storage_bundle_mapping_form';
-      if (function_exists($function)) {
-        $store_form = $function($form, $form_state, $selected_term, $submit_disabled);
-        $form['storage'][$store_type] = $store_form;
-      }
+/**
+ * Builds step1 of the tripal_admin_add_type_form()
+ */
+function tripal_admin_add_type_form_step3(&$form, &$form_state, $stores, $selected_term, $default_store) {
+  $default_store = $form_state['values']['store_select'];
+  
+  $selected_store_module = $stores[$default_store]['module'];
+  
+  $form['store_settings'] = [
+    '#type' => 'fieldset',
+    '#title' => t('Step 3: Storage Settings'),
+    '#description' => '',
+  ];
 
-      // Add in the button for the cases of no terms or too many.
-      $form['submit_button'] = array(
-        '#type' => 'submit',
-        '#value' => t('Create content type'),
-        '#name' => 'use_cvterm',
-        '#disabled' => $submit_disabled,
-      );
-    }
+  $function = $selected_store_module . '_field_storage_bundle_mapping_form';
+  if (function_exists($function)) {
+    $store_form = $function($form, $form_state, $selected_term, $submit_disabled);
+    $form['store_settings'][$default_store] = $store_form;
   }
-
-
-  return $form;
+  
+  // Add in the button for the cases of no terms or too many.
+  $form['submit_button'] = array(
+    '#type' => 'submit',
+    '#value' => t('Create content type'),
+    '#name' => 'create-content',
+    '#disabled' => $submit_disabled,
+  );
 }
 /**
  * Implements an AJAX callback for the tripal_chado_vocab_select_term_form.
  */
 function tripal_admin_add_type_form_ajax_callback($form, $form_state) {
+
   return $form;
 }
 /**
@@ -755,50 +767,52 @@ function tripal_admin_add_type_form_ajax_callback($form, $form_state) {
 function tripal_admin_add_type_form_validate($form, &$form_state) {
   $stores = module_invoke_all('vocab_storage_info');
   $store_select = (isset($form_state['values']['store_select'])) ? $form_state['values']['store_select'] : NULL;
+  $clicked_button = $form_state['clicked_button']['#name'];
+  
+  // Don't do validation on an ajax callback.
+  if (array_key_exists('#ajax', $form_state['triggering_element'])) {
+    return;
+  }
 
-  if (array_key_exists('clicked_button', $form_state) and
-      $form_state['clicked_button']['#name'] =='use_cvterm') {
-
-    $cvterm_id = NULL;
-
-    // Make sure we have a cvterm selected
-    $num_selected = 0;
-    foreach ($form_state['values'] as $key => $value) {
-      $matches = array();
-      if (preg_match("/^term-(\d+)$/", $key, $matches) and
-          $form_state['values']['term-' . $matches[1]]) {
-        $cvterm_id = $matches[1];
-        $term = chado_generate_var('cvterm', array('cvterm_id' => $cvterm_id));
-        $num_selected++;
-      }
+  if ($clicked_button =='step1-continue') {
+    $form_state['rebuild'] = TRUE;
+    $form_state['stage'] = 'step2';   
+    
+    $selected = tripal_get_term_lookup_form_result($form, $form_state);
+    if (count($selected) == 0) {
+      form_set_error('term_match][term_name', 'Please select a vocabulary term.');
     }
-
-    if ($num_selected == 0) {
-      form_set_error('', 'Please select at least one term.');
+    if (count($selected) > 1) {
+      form_set_error('term_match][term_name', 'Please select only one vocabulary term.');
     }
-    else if ($num_selected > 1) {
-      form_set_error('term-' . $cvterm_id, 'Please select only one term from the list below.');
+    
+  }
+  if ($clicked_button =='step1-return') {
+    $form_state['rebuild'] = TRUE;
+    $form_state['stage'] = 'step1';
+  }
+  if ($clicked_button =='step2-continue') {
+    
+    if (!$store_select) {
+      form_set_error('store_select', 'Please select only one vocabulary term.');
     }
-    else {
-      // Add the term to the form state so we can access it later.
-      $form_state['term']['vocabulary'] = $term->dbxref_id->db_id->name;
-      $form_state['term']['accession'] = $term->dbxref_id->accession;
-      $form_state['term']['term_name'] = $term->name;
-
-      // Call the submit hook for this form for the storage method that
-      // will be responsible for this cotent type.
-      $stores = module_invoke_all('vocab_storage_info');
-      $selected_store_module = $stores[$store_select]['module'];
-      $selected_term = $form_state['term'];
-
-      $function = $selected_store_module . '_field_storage_bundle_mapping_form_validate';
-      if (function_exists($function)) {
-        $function($form, $form_state, $selected_term);
-      }
+    
+    $form_state['rebuild'] = TRUE;
+    $form_state['stage'] = 'step3';
+  }
+  
+  if ($clicked_button == 'create-content') {           
+    // Call the submit hook for this form for the storage method that
+    // will be responsible for this cotent type.
+    $stores = module_invoke_all('vocab_storage_info');
+    $selected_store_module = $stores[$store_select]['module'];
+    $selected_term = $form_state['values']['term'];
+
+    $function = $selected_store_module . '_field_storage_bundle_mapping_form_validate';
+    if (function_exists($function)) {
+      $function($form, $form_state, $selected_term);
     }
   }
-  // For any other button click it's an AJAX call and we just want to reubild
-  // the form.
   else {
     $form_state['rebuild'] = TRUE;
   }
@@ -809,16 +823,16 @@ function tripal_admin_add_type_form_validate($form, &$form_state) {
 function tripal_admin_add_type_form_submit($form, &$form_state) {
   $vocabulary = '';
   $accession = '';
-  if (array_key_exists('term', $form_state)) {
-    $selected_term = $form_state['term'];
+  if (array_key_exists('term', $form_state['values'])) {
+    $selected_term = $form_state['values']['term'];
     $store_select = $form_state['values']['store_select'];
-    $vocabulary = array_key_exists('vocabulary', $selected_term) ? $selected_term['vocabulary'] : '';
-    $accession = array_key_exists('accession', $selected_term) ? $selected_term['accession'] : '';
-    $term_name = array_key_exists('term_name', $selected_term) ? $selected_term['term_name'] : '';
+    $vocabulary = $selected_term->dbxref_id->db_id->name;
+    $accession = $selected_term->dbxref_id->accession;
+    $term_name = $selected_term->name;
 
     // Before we try to add this type, check to see if it already exists
     // as a bundle.
-    $term = tripal_load_term_entity(array('vocabulary' => $vocabulary, 'accession' => $accession));
+    $term = tripal_load_term_entity(['vocabulary' => $vocabulary, 'accession' => $accession]);
     if (!$term) {
 
       // Call the submit hook for this form for the storage method that
@@ -853,7 +867,7 @@ function tripal_admin_add_type_form_submit($form, &$form_state) {
       }
     }
     else {
-      drupal_set_message("The term '$accession' already exists as a content type.", 'warning');
+      drupal_set_message("The term ". $term->name . " (" . $vocabulary . ':' . $accession. ") already exists as a content type.", 'warning');
     }
   }
 }

tripal/includes/TripalEntityController.inc

@@ -91,7 +91,9 @@ class TripalEntityController extends EntityAPIController {
       }
     }
     catch (Exception $e) {
-      $transaction->rollback();
+      if ($transaction) {
+        $transaction->rollback();
+      }
       watchdog_exception('tripal', $e);
       throw $e;
       return FALSE;
@@ -468,7 +470,7 @@ class TripalEntityController extends EntityAPIController {
     catch (Exception $e) {
       $transaction->rollback();
       watchdog_exception('tripal', $e);
-      drupal_set_message("Could not save the entity: " . $e->getMessage(), "error");
+      drupal_set_message("Could not save the TripalEntity: " . $e->getMessage(), "error");
       return FALSE;
     }
   }

tripal/includes/TripalJob.inc

@@ -595,6 +595,15 @@ class TripalJob {
    * Logging works regardless if the job uses a transaction. If the
    * transaction must be rolled back to to an error the error messages will
    * persist.
+   * 
+   * If a function can be executed by the Tripal job system (and hence the
+   * job object is passed in) then you can directly use this function to
+   * log messages. However, if the function can be run via drush on the 
+   * command-line, consider using the tripal_report_error() function which can
+   * accept a job object as an $option and will print to both the terminal
+   * and to the job object.  If you use the tripal_report_error() be sure
+   * to set the 'watchdog' option only if you need log messages also going 
+   * to the watchdog.
    *
    * @param $message
    *   The message to store in the log. Keep $message translatable by not

tripal/includes/TripalTermController.inc

@@ -105,7 +105,7 @@ class TripalTermController extends EntityAPIController {
     catch (Exception $e) {
       $transaction->rollback();
       watchdog_exception('tripal_entity', $e);
-      drupal_set_message("Could not save the entity:" . $e->getMessage(), "error");
+      drupal_set_message("Could not save the TripalTerm:" . $e->getMessage(), "error");
       return FALSE;
     }
   }

tripal/includes/TripalVocabController.inc

@@ -106,7 +106,7 @@ class TripalVocabController extends EntityAPIController {
     catch (Exception $e) {
       $transaction->rollback();
       watchdog_exception('tripal_entity', $e);
-      drupal_set_message("Could not save the entity:" . $e->getMessage(), "error");
+      drupal_set_message("Could not save the TripalVocab:" . $e->getMessage(), "error");
       return FALSE;
     }
   }

tripal/includes/tripal.jobs.inc

@@ -279,12 +279,21 @@ function tripal_jobs_view($job_id) {
   }
 
   // build the links
-  $links  = l('Return to jobs list', "admin/tripal/tripal_jobs/") . ' | ';
-  $links .= l('Re-run this job', "admin/tripal/tripal_jobs/rerun/" . $job->job_id) . ' | ';
+  $items = [];
+  $items[] = l('Return to jobs list', "admin/tripal/tripal_jobs/");
+  $items[] = l('Re-run this job', "admin/tripal/tripal_jobs/rerun/" . $job->job_id);
   if ($job->start_time == 0 and $job->end_time == 0) {
-    $links .= l('Cancel this job', "admin/tripal/tripal_jobs/cancel/" . $job->job_id) . ' | ';
-    $links .= l('Execute this job', "admin/tripal/tripal_jobs/execute/".$job->job_id);
+    $items[] = l('Cancel this job', "admin/tripal/tripal_jobs/cancel/" . $job->job_id);
+    $items[] = l('Execute this job', "admin/tripal/tripal_jobs/execute/".$job->job_id);
   }
+  $links = theme_item_list([    
+    'items' => $items,
+    'title' => '',
+    'type' => 'ul',
+    'attributes' => [
+      'class' => ['action-links'],
+    ],
+  ]);
 
   // make our start and end times more legible
   $job->submit_date = tripal_get_job_submit_date($job);
@@ -320,7 +329,7 @@ function tripal_jobs_view($job_id) {
 
   $content['links'] = array(
     '#type' => 'markup',
-    '#markup' => '<p>' . substr($links, 0, -2) . '</p>',
+    '#markup' => $links,
   );
   $content['job_title'] = array(
     '#type' => 'item',
@@ -362,7 +371,7 @@ function tripal_jobs_view($job_id) {
   );
   $content['log_fset']['job_logs'] = array(
     '#type' => 'markup',
-    '#markup' => '<pre>' . $job->error_msg . '</pre>',
+    '#markup' => '<pre class="tripal-job-logs">' . $job->error_msg . '</pre>',
   );
   return $content;
 }

tripal/theme/css/tripal.css

@@ -103,4 +103,14 @@ div.messages.tripal-site-admin-only {
    stroke: black;
  }
  
- #tripal-entity-type-chart .x.axis text { display: none; }
+ #tripal-entity-type-chart .x.axis text { display: none; }
+ 
+ .tripal-job-logs {
+  height: 500px;
+  overflow: scroll;
+  font-framily: Courier New, monospace;
+}
+
+.item-list .action-links li {
+	list-style-type: none;
+} 

tripal_chado/api/modules/tripal_chado.cv.api.inc

@@ -414,7 +414,8 @@ function chado_update_cvtermpath($cv_id, $job_id = NULL){
     // Rollback any database changes
     $transaction->rollback();
     throw $e;
-  } finally {
+  } 
+  finally {
     // Set the database back
     chado_set_active($prev_db);
   }

tripal_chado/api/modules/tripal_chado.module.DEPRECATED.api.inc

@@ -1365,7 +1365,7 @@ function tripal_import_pub_by_dbxref($pub_dbxref, $do_contact = false, $do_updat
  * @ingroup tripal_chado_module_DEPRECATED_api
  */
 function tripal_execute_active_pub_importers($report_email = false, $do_update = false) {
-  return chado_execute_active_pub_importers($report_email, $do_update);
+  return chado_execute_active_pub_importers($report_email, TRUE, $do_update);
 }
 
 /**

tripal_chado/api/modules/tripal_chado.pub.api.inc

@@ -331,13 +331,21 @@ function chado_autocomplete_pub($string = '') {
  * @param $do_contact
  *   Set to TRUE if authors should automatically have a contact record added
  *   to Chado.
+ * @param $publish
+ *   Set to TRUE if publications should be published after import.  For Tripal
+ *   v3 this value can be set to the string 'sync' or 'both' in the event that
+ *   the site is in "legacy" mode.  Setting this value to 'sync' will create
+ *   nodes, setting to 'both' will create nodes and entities.  If set to TRUE
+ *   only entities are created.
  * @param $do_update
  *   If set to TRUE then the publication will be updated if it already exists
  *   in the database.
  *
  * @ingroup tripal_pub_api
  */
-function chado_import_pub_by_dbxref($pub_dbxref, $do_contact = FALSE, $do_update = TRUE) {
+function chado_import_pub_by_dbxref($pub_dbxref, $do_contact = FALSE, 
+    $publish = TRUE, $do_update = TRUE) {
+    
   $num_to_retrieve = 1;
   $pager_id = 0;
   $page = 0;
@@ -345,10 +353,21 @@ function chado_import_pub_by_dbxref($pub_dbxref, $do_contact = FALSE, $do_update
   $pub_id = NULL;
 
   module_load_include('inc', 'tripal_chado', 'includes/loaders/tripal_chado.pub_importers');
-
-  print "\nNOTE: Loading of publications is performed using a database transaction. \n" .
-      "If the load fails or is terminated prematurely then the entire set of \n" .
-      "insertions/updates is rolled back and will not be found in the database\n\n";
+  
+  // These are options for the tripal_report_error function. We do not
+  // want to log messages to the watchdog but we do for the job and to
+  // the terminal
+  $message_type = 'pub_import';
+  $message_opts = [
+    'watchdog' == FALSE,
+    'print' => TRUE,
+  ];
+  
+  $message = "Importing of publications is performed using a database transaction. " .
+    "If the load fails or is terminated prematurely then the entire set of " .
+    "deletions is rolled back and will not be found in the database";
+  tripal_report_error($message_type, TRIPAL_INFO, $message, [], $message_opts);
+  
 
   $transaction = db_transaction();
   try {
@@ -370,27 +389,15 @@ function chado_import_pub_by_dbxref($pub_dbxref, $do_contact = FALSE, $do_update
       );
       $remote_db = $criteria['remote_db'];
       $results = tripal_get_remote_pubs($remote_db, $criteria, $num_to_retrieve, $page);
-      $pubs          = $results['pubs'];
-      $search_str    = $results['search_str'];
+      $pubs = $results['pubs'];
+      $search_str = $results['search_str'];
       $total_records = $results['total_records'];
       tripal_pub_add_publications($pubs, $do_contact, $do_update);
     }
 
-    // For backwards compatibility check to see if the legacy pub module
-    // is enabled. If so, then sync the nodes.
-    if (module_exists('tripal_pub')) {
-
-      // Sync the newly added publications with Drupal.
-      print "Syncing publications with Drupal...\n";
-      chado_node_sync_records('pub');
-
-      // If any of the importers wanted to create contacts from the authors 
-      // then sync them.
-      if($do_contact) {
-        print "Syncing contacts with Drupal...\n";
-        chado_node_sync_records('contact');
-      }
-    }
+    // Publish as requested by the caller.
+    _chado_execute_pub_importer_publish($publish, NULL, $message_type, $message_opts);
+    
   }
   catch (Exception $e) {
     $transaction->rollback();
@@ -407,108 +414,212 @@ function chado_import_pub_by_dbxref($pub_dbxref, $do_contact = FALSE, $do_update
  * @param $report_email
  *   A list of email address, separated by commas, that should be notified
  *   once importing has completed.
+ * @param $publish
+ *   Set to TRUE if publications should be published after import.  For Tripal
+ *   v3 this value can be set to the string 'sync' or 'both' in the event that
+ *   the site is in "legacy" mode.  Setting this value to 'sync' will create
+ *   nodes, setting to 'both' will create nodes and entities.  If set to TRUE
+ *   only entities are created.
  * @param $do_update
  *   If set to TRUE then publications that already exist in the Chado database
  *   will be updated, whereas if FALSE only new publications will be added.
  *
  * @ingroup tripal_pub_api
  */
-function chado_execute_active_pub_importers($report_email = FALSE, $do_update = FALSE) {
-  $num_to_retrieve = 100;
-  $page = 0;
+function chado_execute_active_pub_importers($report_email = FALSE, 
+  $publish = TRUE, $do_update = FALSE) {
+
+  $report = [];
+  $report['error'] = [];
+  $report['inserted'] = [];
+  $report['skipped'] = [];
+  $report['updated'] = [];
+    
+  // Get all of the loaders.
+  $args = array();
+  $sql = "SELECT * FROM {tripal_pub_import} WHERE disabled = 0 ";
+  $importers = db_query($sql, $args);
+  $do_contact = FALSE;
+  while ($import = $importers->fetchObject()) {
+    $importer_report = chado_execute_pub_importer($import->pub_import_id, $publish, $do_update);
+    foreach ($importer_report as $action => $pubs) {
+      $report[$action] = array_merge($report[$action], $pubs);
+    }
+  }
 
-  print "\nNOTE: Loading of publications is performed using a database transaction. \n" .
-      "If the load fails or is terminated prematurely then the entire set of \n" .
-      "insertions/updates is rolled back and will not be found in the database\n\n";
+  $site_email = variable_get('site_mail', '');
+  $params = array(
+    'report' => $report
+  );
+  drupal_mail('tripal_chado', 'import_report', $report_email, language_default(), $params, $site_email, TRUE);
+  print "Done.\n";
+}
 
+
+/**
+ * Imports all publications for a given publication import setup.
+ *
+ * @param $import_id
+ *   The ID of the import setup to use
+ * @param $publish
+ *   Set to TRUE if publications should be published after import.  For Tripal
+ *   v3 this value can be set to the string 'sync' or 'both' in the event that
+ *   the site is in "legacy" mode.  Setting this value to 'sync' will create
+ *   nodes, setting to 'both' will create nodes and entities.  If set to TRUE
+ *   only entities are created.
+ * @param $do_update
+ *   If set to TRUE then publications that already exist in the Chado database
+ *   will be updated, whereas if FALSE only new publications will be added.
+ * @param $job
+ *   The jobs management object for the job if this function is run as a job. 
+ *   This argument is added by Tripal during a job run and is not needed if
+ *   this function is run directly.
+ *   
+ * @return
+ *   Returns an array containing the number of publications that were
+ *   inserted, updated, skipped and which had an error during import.
+ *
+ * @ingroup tripal_pub
+ */
+function chado_execute_pub_importer($import_id, $publish = TRUE, 
+  $do_update = FALSE, $job = NULL) {
+  
+  // Holds the list of imported pubs which includes their ID and Citation.
+  $report = [];
+  $report['error'] = [];
+  $report['inserted'] = [];
+  $report['skipped'] = [];
+  $report['updated'] = [];
+  
+  // These are options for the tripal_report_error function. We do not
+  // want to log messages to the watchdog but we do for the job and to
+  // the terminal
+  $message_type = 'pub_import';
+  $message_opts = [
+    'watchdog' == FALSE,
+    'job' => $job,
+    'print' => TRUE,
+  ];
+  
+  $message = "Importing of publications for this importer is performed using a database transaction. " .
+    "If the load fails or is terminated prematurely then the entire set of " .
+    "deletions is rolled back and will not be found in the database";  
+  tripal_report_error($message_type, TRIPAL_INFO, $message, [], $message_opts);
+  
+   
   // start the transaction
   $transaction = db_transaction();
-
+  
   try {
-    // Get all of the loaders.
-    $args = array();
-    $sql = "SELECT * FROM {tripal_pub_import} WHERE disabled = 0 ";
-    $results = db_query($sql, $args);
+    $page = 0;
     $do_contact = FALSE;
-    $reports = array();
-    foreach ($results as $import) {
-      $page = 0;
-      print "Executing importer: '" . $import->name . "'\n";
-      // Keep track if any of the importers want to create contacts from authors.
-      if ($import->do_contact == 1) {
-        $do_contact = TRUE;
-      }
-      $criteria = unserialize($import->criteria);
-      $remote_db = $criteria['remote_db'];
-      do {
-        // Retrieve the pubs for this page. We'll retreive 100 at a time.
-        $results = tripal_get_remote_pubs($remote_db, $criteria, $num_to_retrieve, $page);
-        $pubs = $results['pubs'];
-        $reports[$import->name] = tripal_pub_add_publications($pubs, $import->do_contact, $do_update);
-        $page++;
+    $num_to_retrieve = 100;
+    
+    // get all of the loaders
+    $args = array(':import_id' => $import_id);
+    $sql = "SELECT * FROM {tripal_pub_import} WHERE pub_import_id = :import_id ";
+    $import = db_query($sql, $args)->fetchObject();
+    
+    tripal_report_error($message_type, TRIPAL_INFO, 
+      "Executing Importer: !name.", ['!name' => $import->name], $message_opts);
+    
+    $criteria = unserialize($import->criteria);
+    $remote_db = $criteria['remote_db'];
+    $total_pubs = 0;
+    
+    // Loop until we have a $pubs array that does not have
+    // our requested numer of records.  This means we've hit the end    
+    do {
+      // retrieve the pubs for this page. We'll retreive 100 at a time
+      tripal_report_error($message_type, TRIPAL_INFO,
+        "Querying !remote_db for up to !num pubs that match the criteria.", 
+        ['!num' => $num_to_retrieve, '!remote_db' => $remote_db], $message_opts);
+      $results = tripal_get_remote_pubs($remote_db, $criteria, $num_to_retrieve, $page);
+      $pubs = $results['pubs'];
+      $num_pubs = $results['total_records'];
+      $total_pubs += $num_pubs;
+      tripal_report_error($message_type, TRIPAL_INFO,
+        "Found %num publications.",
+        ['%num' => $num_pubs], $message_opts);
+      
+      $subset_report = tripal_pub_add_publications($pubs, $import->do_contact, $do_update, $job);
+      foreach ($subset_report as $action => $pubs) {
+        $report[$action] = array_merge($report[$action], $pubs);
       }
-      // Continue looping until we have a $pubs array that does not have
-      // our requested numer of records.  This means we've hit the end.
-      while (count($pubs) == $num_to_retrieve);
+      $page++;
     }
-
-    // Sync the newly added publications with Drupal. If the user
-    // requested a report then we don't want to print any syncing information
-    // so pass 'FALSE' to the sync call.
-    // For backwards compatibility check to see if the legacy pub module
-    // is enabled. If so, then sync the nodes.
-    if (module_exists('tripal_pub')) {
-      print "Syncing publications with Drupal...\n";
-      chado_node_sync_records('pub');
+    while (count($pubs) == $num_to_retrieve);
+    
+    // Publish as requested by the caller.
+    _chado_execute_pub_importer_publish($publish, $job, $message_type, $message_opts);
+    
+    if ($job) {
+      $job->setProgress(100);
     }
+  }
+  catch (Exception $e) {
+    $transaction->rollback();
+    watchdog_exception('T_pub_import', $e);
+    tripal_report_error($message_type, TRIPAL_ERROR,
+      "Rolling back database changes... !message", 
+      ['!message' => $e->getMessage()], $message_opts);
+    return FALSE;
+  }
+  
+  tripal_report_error($message_type, TRIPAL_INFO,
+    "Done.", [], $message_opts);
+  
+  return $report;
+}
 
-    // Iterate through each of the reports and generate a final report with HTML
-    // links.
-    $HTML_report = '';
-    if ($report_email) {
-      $HTML_report .= "<html>";
-      global $base_url;
-      foreach ($reports as $importer => $report) {
-        $total = count($report['inserted']);
-        $HTML_report .= "<b>$total new publications from importer: $importer</b><br><ol>\n";
-        foreach ($report['inserted'] as $pub) {
-          $item = $pub['Title'];
-          if (array_key_exists('pub_id', $pub)) {
-            $item = l($pub['Title'], "$base_url/pub/" . $pub['pub_id']);
-          }
-          $HTML_report .= "<li>$item</li>\n";
-        }
-        $HTML_report .= "</ol>\n";
-      }
-      $HTML_report .= "</html>";
-      $site_email = variable_get('site_mail', '');
-      $params = array(
-        'message' => $HTML_report
-      );
-      drupal_mail('tripal_pub', 'import_report', $report_email, language_default(), $params, $site_email, TRUE);
-    }
 
-    // For backwards compatibility check to see if the legacy pub module
-    // is enabled. If so, then sync the nodes.
+/**
+ * A helper function to dermine if imported publications should be published.
+ * 
+ * It supports backwards compatibility with Tripal v2 legacy mode.
+ * 
+ * @param $publish
+ *   Set to TRUE if publications should be published after import.  For Tripal
+ *   v3 this value can be set to the string 'sync' or 'both' in the event that
+ *   the site is in "legacy" mode.  Setting this value to 'sync' will create
+ *   nodes, setting to 'both' will create nodes and entities.  If set to TRUE
+ *   only entities are created.
+ */
+function _chado_execute_pub_importer_publish($publish, $job, $message_type, $message_opts) {
+    
+  // If the user wants to publish then do so.
+  if ($publish === TRUE or $publish === 'both') {
+    
+    // Get the bundle for the Publication content type.
+    $bundle_term = tripal_load_term_entity(['vocabulary' => 'TPUB', 'accession' => '0000002']);
+    $bundle = tripal_load_bundle_entity(['term_id' => $bundle_term->id]);
+    if ($bundle) {
+      tripal_report_error($message_type, TRIPAL_INFO,
+        "Publishing publications with Drupal...", [], $message_opts);
+      chado_publish_records(['bundle_name' => $bundle->name], $job);
+    }
+    // Note: we won't publish contacts as Tripal v2 did because there is
+    // no consisten way to do that. Each site my use a different term for
+    // different contact content types (e.g. all as one 'Contact' type or
+    // specific such as 'Person', 'Organization', etc.).
+  }
+  
+  // For backwords compatibility with legacy module do a sync.
+  if ($publish === 'sync' or $publish === 'both') {
     if (module_exists('tripal_pub')) {
-      // If any of the importers wanted to create contacts from the authors then 
-      // sync them.
-      if($do_contact) {
-        print "Syncing contacts with Drupal...\n";
+      tripal_report_error($message_type, TRIPAL_INFO,
+        "Syncing publications with Drupal...", [], $message_opts);
+      chado_node_sync_records('pub');
+      if($import->do_contact) {
+        tripal_report_error($message_type, TRIPAL_INFO,
+          "Syncing contacts with Drupal...", [], $message_opts);
         chado_node_sync_records('contact');
       }
     }
   }
-  catch (Exception $e) {
-    $transaction->rollback();
-    print "\n"; // make sure we start errors on new line
-    watchdog_exception('T_pub_import', $e);
-    print "FAILED: Rolling back database changes...\n";
-    return;
-  }
-  print "Done.\n";
 }
 
+
 /**
  * Updates publication records.
  *
@@ -528,14 +639,32 @@ function chado_execute_active_pub_importers($report_email = FALSE, $do_update =
  *   The name of the remote database to update.  If this value is provided and
  *   no dbxref then all of the publications currently in the Chado database
  *   for this remote database will be updated.
- *
+ * @param $publish
+ *   Set to TRUE if publications should be published after import.  For Tripal
+ *   v3 this value can be set to the string 'sync' or 'both' in the event that
+ *   the site is in "legacy" mode.  Setting this value to 'sync' will create
+ *   nodes, setting to 'both' will create nodes and entities.  If set to TRUE
+ *   only entities are created.
+
  * @ingroup tripal_pub_api
  */
-function chado_reimport_publications($do_contact = FALSE, $dbxref = NULL, $db = NULL) {
-
-  print "\nNOTE: Loading of publications is performed using a database transaction. \n" .
-      "If the load fails or is terminated prematurely then the entire set of \n" .
-      "insertions/updates is rolled back and will not be found in the database\n\n";
+function chado_reimport_publications($do_contact = FALSE, $dbxref = NULL, 
+    $db = NULL, $publish = TRUE) {
+
+  // These are options for the tripal_report_error function. We do not
+  // want to log messages to the watchdog but we do for the job and to
+  // the terminal
+  $message_type = 'pub_import';
+  $message_opts = [
+    'watchdog' == FALSE,
+    'print' => TRUE,
+  ];
+  
+  $message = "Importing of publications for this importer is performed using a database transaction. " .
+    "If the load fails or is terminated prematurely then the entire set of " .
+    "deletions is rolled back and will not be found in the database";
+  tripal_report_error($message_type, TRIPAL_INFO, $message, [], $message_opts);
+  
   $transaction = db_transaction();
   try {
 
@@ -543,10 +672,10 @@ function chado_reimport_publications($do_contact = FALSE, $dbxref = NULL, $db =
     // databases.
     $sql = "
       SELECT DB.name as db_name, DBX.accession
-      FROM pub P
-        INNER JOIN pub_dbxref PDBX ON P.pub_id = PDBX.pub_id
-        INNER JOIN dbxref DBX      ON DBX.dbxref_id = PDBX.dbxref_id
-        INNER JOIN db DB           ON DB.db_id = DBX.db_id
+      FROM {pub} P
+        INNER JOIN {pub_dbxref} PDBX ON P.pub_id = PDBX.pub_id
+        INNER JOIN {dbxref} DBX      ON DBX.dbxref_id = PDBX.dbxref_id
+        INNER JOIN {db} DB           ON DB.db_id = DBX.db_id
     ";
     $args = array();
     if ($dbxref and preg_match('/^(.*?):(.*?)$/', $dbxref, $matches)) {
@@ -591,32 +720,22 @@ function chado_reimport_publications($do_contact = FALSE, $dbxref = NULL, $db =
           ),
         ),
       );
-      $pubs = tripal_get_remote_pubs($remote_db, $search, 1, 0);
+      $qresults = tripal_get_remote_pubs($remote_db, $search, 1, 0);
+      $pubs = $qresults['pubs'];      
       tripal_pub_add_publications($pubs, $do_contact, TRUE);
 
       $i++;
     }
 
-    // For backwards compatibility check to see if the legacy pub module
-    // is enabled. If so, then sync the nodes.
-    if (module_exists('tripal_pub')) {
-      // Sync the newly added publications with Drupal.
-      print "Syncing publications with Drupal...\n";
-      chado_node_sync_records('pub');
-
-      // If the caller wants to create contacts then we should sync them.
-      if ($do_contact) {
-        print "Syncing contacts with Drupal...\n";
-        chado_node_sync_records('contact');
-      }
-    }
+    // Publish as requested by the caller.
+    _chado_execute_pub_importer_publish($publish, NULL, $message_type, $message_opts);
   }
   catch (Exception $e) {
     $transaction->rollback();
-    print "\n"; // make sure we start errors on new line
     watchdog_exception('T_pub_import', $e);
-    print "FAILED: Rolling back database changes...\n";
-    return;
+    tripal_report_error($message_type, TRIPAL_ERROR,
+      "Rolling back database changes... !message",
+      ['!message' => $e->getMessage()], $message_opts);return;
   }
   print "Done.\n";
 }

tripal_chado/api/tripal_chado.api.inc

@@ -26,10 +26,10 @@
  * @param $values
  *   A key/value associative array that supports the following keys:
  *   - bundle_name:  The name of the the TripalBundle (e.g. bio_data-12345).
- * @param $job_id
- *   (Optional) The numeric job ID as provided by the Tripal jobs system. There
- *   is no need to specify this argument if this function is being called
- *   outside of the jobs systems.
+ * @param $job
+ *   The jobs management object for the job if this function is run as a job. 
+ *   This argument is added by Tripal during a job run and is not needed if
+ *   this function is run directly.
  *
  * @return boolean
  *   TRUE if all of the records of the given bundle type were published, and
@@ -37,23 +37,27 @@
  *
  * @ingroup tripal_chado_api
  */
-function chado_publish_records($values, $job_id = NULL) {
-
+function chado_publish_records($values, $job = NULL) {
+ 
   // Used for adding runtime to the progress report.
   $started_at = microtime(true);
 
-  // We want the job object in order to report progress.
-  if (is_object($job_id)) {
-    $job = $job_id;
-  }
-  if (is_numeric($job_id)) {
+  // We want the job object in order to report progress with the job.
+  if (is_numeric($job)) {
+    $job_id = $job;
     $job = new TripalJob();
     $job->load($job_id);
   }
-  $report_progress = FALSE;
-  if (is_object($job)) {
-    $report_progress = TRUE;
-  }
+  
+  // These are options for the tripal_report_error function. We do not
+  // want to log messages to the watchdog but we do for the job and to
+  // the terminal
+  $message_type = 'publish_records';
+  $message_opts = [
+    'watchdog' == FALSE,
+    'job' => $job,
+    'print' => TRUE,
+  ];
 
   // Start an array for caching objects to save performance.
   $cache = array();
@@ -62,7 +66,7 @@ function chado_publish_records($values, $job_id = NULL) {
   if (!array_key_exists('bundle_name', $values) or !$values['bundle_name']) {
     tripal_report_error('tripal_chado', TRIPAL_ERROR,
       "Could not publish record: @error",
-      array('@error' => 'The bundle name must be provided'));
+     ['@error' => 'The bundle name must be provided'], $message_opts);
     return FALSE;
   }
 
@@ -81,9 +85,9 @@ function chado_publish_records($values, $job_id = NULL) {
   $bundle = tripal_load_bundle_entity(array('name' => $bundle_name));
   $cache['bundle'] = $bundle;
   if (!$bundle) {
-    tripal_report_error('tripal_chado', TRIPAL_ERROR,
-        "Unknown bundle. Could not publish record: @error",
-        array('@error' => 'The bundle name must be provided'));
+    tripal_report_error($message_type, TRIPAL_ERROR,
+      "Unknown bundle. Could not publish record: @error",
+      ['@error' => 'The bundle name must be provided'], $message_opts);
     return FALSE;
   }
   $chado_entity_table = chado_get_bundle_entity_table($bundle);
@@ -94,14 +98,14 @@ function chado_publish_records($values, $job_id = NULL) {
     ->condition('bundle_id', $bundle->id)
     ->execute()
     ->fetchObject();
-  if(!$chado_bundle) {
+  if (!$chado_bundle) {
     tripal_report_error('tripal_chado', TRIPAL_ERROR,
         "Cannot find mapping of bundle to Chado tables. Could not publish record.");
     return FALSE;
   }
 
   // Load the term for use in setting the alias for each entity created.
-  $term = entity_load('TripalTerm', array('id' => $entity->term_id));
+  $term = entity_load('TripalTerm', array('id' => $bundle->term_id));
   $cache['term'] = $term;
 
   $table = $chado_bundle->data_table;
@@ -202,13 +206,10 @@ function chado_publish_records($values, $job_id = NULL) {
   $sql = "SELECT count(*) as num_records " . $from . $where;
   $result = chado_query($sql, $args);
   $count = $result->fetchField();
-  print "\nThere are $count records to publish.\n";
-
-  print "\nNOTE: publishing records is performed using database transactions. If the job fails\n" .
-          "or is terminated prematurely then the current set of $chunk_size is rolled back with\n" .
-          "no changes to the database. Simply re-run the publishing job to publish any remaining\n".
-          "content after fixing the issue that caused the job to fail.\n\n" .
-          "Also, the following progress only updates every $chunk_size records.\n";
+  
+  tripal_report_error($message_type, TRIPAL_INFO,
+    "There are !count records to publish.",
+    ['!count' => $count], $message_opts);
 
   // Perform the query.
   $sql = $select . $from . $where . ' LIMIT '.$chunk_size;
@@ -220,7 +221,10 @@ function chado_publish_records($values, $job_id = NULL) {
 
     // Update the job status every chunk start.
     // Because this is outside of hte transaction, we can update the admin through the jobs UI.
-    $complete = ($total_published / $count) * 33.33333333;
+    $complete = 0;
+    if ($count > 0) {
+      $complete = ($total_published / $count) * 33.33333333;
+    }
     if ($report_progress) { $job->setProgress(intval($complete * 3)); }
     if ($total_published === 0) {
       printf("%d of %d records. (%0.2f%%) Memory: %s bytes.\r",
@@ -231,13 +235,13 @@ function chado_publish_records($values, $job_id = NULL) {
         $total_published, $count, $complete * 3, number_format(memory_get_usage()), number_format((microtime(true) - $started_at)/60, 2));
     }
 
-    // There is no need to cache transactions since Drupal handles nested transactions
-    // "by performing no transactional operations (as far as the database sees) within
-    // the inner nesting layers". Effectively, Drupal ensures nested trasactions work the
-    // same as passing a transaction through to the deepest level and not starting a new
-    // transaction if we are already in one.
+    // There is no need to cache transactions since Drupal handles nested 
+    // transactions "by performing no transactional operations (as far as the 
+    // database sees) within the inner nesting layers". Effectively, Drupal 
+    // ensures nested trasactions work the same as passing a transaction 
+    // through to the deepest level and not starting a new transaction if we 
+    // are already in one.
     $transaction = db_transaction();
-
     try {
       $i = 0;
       while($record = $records->fetchObject()) {
@@ -290,7 +294,7 @@ function chado_publish_records($values, $job_id = NULL) {
     catch (Exception $e) {
       $transaction->rollback();
       $error = $e->getMessage();
-      tripal_report_error('tripal_chado', TRIPAL_ERROR, "Could not publish record: @error", array('@error' => $error));
+      tripal_report_error($message_type, TRIPAL_ERROR, "Could not publish record: @error", array('@error' => $error));
       drupal_set_message('Failed publishing record. See recent logs for more details.', 'error');
       return FALSE;
     }
@@ -304,7 +308,10 @@ function chado_publish_records($values, $job_id = NULL) {
     unset($transaction);
   }
 
-  drupal_set_message("Succesfully published $total_published " . $bundle->label . " record(s).");
+  tripal_report_error($message_type, TRIPAL_INFO,
+    "Succesfully published %count %type record(s).",
+    ['%count' => $total_published, '%type' => $bundle->label], $message_opts);
+  
   return TRUE;
 }
 

tripal_chado/api/tripal_chado.property.api.inc

@@ -335,7 +335,7 @@ function chado_update_property($record, $property, $options = array()) {
 
   // First see if the property is missing (we can't update a missing property.
   $prop = chado_get_property($record, $property);
-  if (count($prop) == 0) {
+  if (!is_array($prop) or count($prop) == 0) {
     if ($insert_if_missing) {
       return chado_insert_property($record, $property);
     }

tripal_chado/files/tpub.obo

@@ -1266,7 +1266,7 @@ subset: MeSH_Publication_Type
 
 [Term]
 id: TPUB:0000231
-name: Video Audio Media
+name: Video-Audio Media
 def: Used with articles which include video files or clips, or for articles which are entirely video.
 is_a: TPUB:0000015 ! Publication Type
 subset: MeSH_Publication_Type

tripal_chado/includes/TripalFields/schema__additional_type/schema__additional_type_widget.inc

@@ -32,7 +32,8 @@ class schema__additional_type_widget extends ChadoFieldWidget {
       $value = $items[0]['value'];
     }
 
-    if (array_key_exists('values', $form_state) and array_key_exists($field_name, $form_state['values'])) {
+    if (array_key_exists('values', $form_state) and array_key_exists($field_name, $form_state['values']) and 
+        array_key_exists('value', $form_state['values'][$field_name][$langcode][$delta])) {
       $type_id = $form_state['values'][$field_name][$langcode][$delta]['value'];
     }
 

tripal_chado/includes/loaders/tripal_chado.pub_importers.inc

@@ -367,7 +367,7 @@ function tripal_pub_importer_setup_form($form, &$form_state = NULL, $pub_import_
   );
   $dbs = chado_select_record('db', array('*'), $values);
   foreach ($dbs as $index => $db) {
-    $remote_dbs[$db->name] = $db->description;
+    $remote_dbs[$db->name] = $db->description ? $db->description : $db->name;
   };
   // use PubMed as the default
   if (!$remote_db) {
@@ -379,7 +379,7 @@ function tripal_pub_importer_setup_form($form, &$form_state = NULL, $pub_import_
    */
   unset($remote_dbs['AGL']);
   $form['themed_element']['remote_db'] = array(
-    '#title' => t('Remote Database'),
+    '#title' => t('Source'),
     '#type' => 'select',
     '#options' => $remote_dbs,
     '#default_value' => $remote_db,
@@ -739,7 +739,7 @@ function tripal_pub_importer_setup_form_submit($form, &$form_state) {
         // if the user wants to do the import now then do it (may time out
         // for long jobs)
         if ($form_state['values']['op'] == 'Save & Import Now') {
-          tripal_execute_pub_importer($record['pub_import_id']);
+          chado_execute_pub_importer($record['pub_import_id']);
         }
         drupal_goto('admin/tripal/loaders/pub');
       }
@@ -844,11 +844,11 @@ function tripal_pub_importer_submit_job($import_id) {
   $sql = "SELECT * FROM {tripal_pub_import} WHERE pub_import_id = :import_id ";
   $import = db_query($sql, $args)->fetchObject();
 
-  $args = array($import_id);
+  $args = array($import_id, TRUE, FALSE);
   $includes = array();
   $includes[] = module_load_include('inc', 'tripal_chado', 'includes/loaders/tripal_chado.pub_importers');
   tripal_add_job("Import publications $import->name", 'tripal_chado',
-    'tripal_execute_pub_importer', $args, $user->uid, 10, $includes);
+    'chado_execute_pub_importer', $args, $user->uid, 10, $includes);
 
   drupal_goto('admin/tripal/loaders/pub');
 }
@@ -883,6 +883,10 @@ function tripal_pub_importer_delete($import_id) {
  * @param $update
  *   If set to TRUE then publications that already exist in the Chado database
  *   will be updated, whereas if FALSE only new publications will be added
+ * @param $job
+ *   The jobs management object for the job if this function is run as a job. 
+ *   This argument is added by Tripal during a job run and is not needed if
+ *   this function is run directly.
  *
  * @return
  *   Returns an array containing the number of publications that were
@@ -890,11 +894,23 @@ function tripal_pub_importer_delete($import_id) {
  *
  * @ingroup tripal_pub
  */
-function tripal_pub_add_publications($pubs, $do_contact, $update = FALSE) {
-  $report = array();
-  $report['error'] = 0;
-  $report['inserted'] = array();
-  $report['skipped'] = array();
+function tripal_pub_add_publications($pubs, $do_contact, $update = FALSE, $job = NULL) {
+  
+  // These are options for the tripal_report_error function. We do not
+  // want to log messages to the watchdog but we do for the job and to
+  // the terminal
+  $message_type = 'pub_import';
+  $message_opts = [
+    'watchdog' == TRUE,
+    'job' => $job,
+    'print' => TRUE,
+  ];
+  
+  $report = [];
+  $report['error'] = [];
+  $report['inserted'] = [];
+  $report['skipped'] = [];
+  $report['updated'] = [];
   $total_pubs = count($pubs);
 
   // iterate through the publications and add each one
@@ -905,7 +921,7 @@ function tripal_pub_add_publications($pubs, $do_contact, $update = FALSE) {
 
     // add the publication to Chado
     $action = '';
-    $pub_id = tripal_pub_add_publication($pub, $action, $do_contact, $update);
+    $pub_id = tripal_pub_add_publication($pub, $action, $do_contact, $update, $job);
     if ($pub_id){
       // add the publication cross reference (e.g. to PubMed)
       if ($pub_id and $pub['Publication Dbxref']) {
@@ -915,11 +931,9 @@ function tripal_pub_add_publications($pubs, $do_contact, $update = FALSE) {
           $dbxref['accession'] = $matches[2];
         }
         else {
-          tripal_report_error(
-            'tripal_pub',
-            TRIPAL_ERROR,
+          tripal_report_error($message_type, TRIPAL_ERROR,
             'Unable to extract the dbxref to be associated with the publication (pub ID=@pub_id) from @dbxref. This reference should be [database-name]:[accession]',
-            array('@pub_id' => $pub_id, '@dbxref' => $pub['Publication Dbxref'])
+            array('@pub_id' => $pub_id, '@dbxref' => $pub['Publication Dbxref'], $message_opts)
           );
         }
         $pub_dbxref = tripal_associate_dbxref('pub', $pub_id, $dbxref);
@@ -929,26 +943,27 @@ function tripal_pub_add_publications($pubs, $do_contact, $update = FALSE) {
 
     switch ($action) {
       case 'error':
-        $report['error']++;
+        $report['error'][] = $pub['Citation'];
         break;
       case 'inserted':
-        $report['inserted'][] = $pub;
+        $report['inserted'][] = $pub['Citation'];
         break;
       case 'updated':
-        $report['updated'][] = $pub;
+        $report['updated'][] = $pub['Citation'];
         break;
       case 'skipped':
-        $report['skipped'][] = $pub;
+        $report['skipped'][] = $pub['Citation'];
         break;
     }
     $i++;
   }
-  print "\n";
   return $report;
 }
 
 /**
- * Adds a new publication to the Chado, along with all properties and
+ * Adds a new publication to Chado.
+ * 
+ * In addition, all properties and
  * database cross-references. If the publication does not already exist
  * in Chado then it is added.  If it does exist nothing is done.  If
  * the $update parameter is TRUE then the publication is updated if it exists.
@@ -959,62 +974,77 @@ function tripal_pub_add_publications($pubs, $do_contact, $update = FALSE) {
  *   This variable will get set to a text value indicating the action that was
  *   performed. The values include 'skipped', 'inserted', 'updated' or 'error'.
  * @param $do_contact
- *   Optional. Set to TRUE if a contact entry should be added to the Chado contact table
- *   for authors of the publication.
+ *   Optional. Set to TRUE if a contact entry should be added to the Chado 
+ *   contact table for authors of the publication.
  * @param $update_if_exists
- *   Optional.  If the publication already exists then this function will return
- *   without adding a new publication.  However, set this value to TRUE to force
- *   the function to pudate the publication using the $pub_details that are provided.
- *
+ *   Optional.  If the publication already exists then this function will 
+ *   return without adding a new publication.  However, set this value to 
+ *   TRUE to force the function to pudate the publication using the 
+ *   $pub_details that are provided.   
+ * @param $job
+ *   The jobs management object for the job if this function is run as a job. 
+ *   This argument is added by Tripal during a job run and is not needed if
+ *   this function is run directly.
  * @return
- *   If the publication already exists, is inserted or updated then the publication
- *   ID is returned, otherwise FALSE is returned. If the publication already exists
- *   and $update_if_exists is not TRUE then the $action variable is set to 'skipped'.
- *   If the publication already exists and $update_if_exists is TRUE and if the update
- *   was successful then $action is set to 'updated'.  Otherwise on successful insert
- *   the $action variable is set to 'inserted'.  If the function failes then the
+ *   If the publication already exists, is inserted or updated then the 
+ *   publication ID is returned, otherwise FALSE is returned. If the 
+ *   publication already exists and $update_if_exists is not TRUE then the 
+ *   $action variable is set to 'skipped'. If the publication already exists 
+ *   and $update_if_exists is TRUE and if the update was successful then 
+ *   $action is set to 'updated'.  Otherwise on successful insert the 
+ *   $action variable is set to 'inserted'.  If the function failes then the
  *   $action variable is set to 'error'
  *
  * @ingroup tripal_pub
  */
-function tripal_pub_add_publication($pub_details, &$action, $do_contact = FALSE, $update_if_exists = FALSE) {
+function tripal_pub_add_publication($pub_details, &$action, $do_contact = FALSE, $update_if_exists = FALSE, $job = NULL) {
   $pub_id = 0;
+  
+  // These are options for the tripal_report_error function. We do not
+  // want to log messages to the watchdog except for errors and to the job and 
+  // to the terminal
+  $message_type = 'pub_import';
+  $message_opts = [
+    'watchdog' == FALSE,
+    'job' => $job,
+    'print' => TRUE,
+  ];
+  $error_opts = [
+    'watchdog' == TRUE,
+    'job' => $job,
+    'print' => TRUE,
+  ];
 
   if (!is_array($pub_details)) {
     return FALSE;
   }
 
-  // before proceeding check to see if the publication already exists. If there is only one match
-  // and the $update_if_exists is NOT set then return FALSE
+  // Before proceeding check to see if the publication already exists. If there 
+  // is only one match and the $update_if_exists is NOT set then return FALSE.
   $pub_ids = chado_publication_exists($pub_details);
 
   if(count($pub_ids) == 1 and !$update_if_exists) {
-    tripal_report_error('tripal_pub', TRIPAL_NOTICE,
-     "There is a publication that is a duplicate of this publication. Cannot continue. It either ".
-     "has a matching Dbxref (e.g. PubMed ID), a non-unique citation or matches on the unique  " .
-     "constraint set by the Tripal publication module configuration page. \nCitation: %title %dbxref.\nMatching Pub id: %ids",
-       array(
-        '%title' => $pub_details['Citation'],
-        '%dbxref' => $pub_details['Publication Dbxref'],
-        '%ids' => implode(",", $pub_ids),
-       )
+    tripal_report_error($message_type, TRIPAL_NOTICE,
+      "The following publication already exists on this site:  %title %dbxref (Matching Pub id: %ids). Skipping.",
+      ['%title' => $pub_details['Citation'],
+       '%dbxref' => $pub_details['Publication Dbxref'],
+       '%ids' => implode(",", $pub_ids)],
+      $message_opts
     );
     $action = 'skipped';
     return FALSE;
   }
-  // if we have more than one matching pub then return an error as we don't know which to update even if
-  // update_if_exists is set to TRUE
+  
+  // If we have more than one matching pub then return an error as we don't 
+  // know which to update even if update_if_exists is set to TRUE.
   if(count($pub_ids) > 1) {
-    tripal_report_error('tripal_pub', TRIPAL_NOTICE,
-      "There are %num publications that are duplicates of this publication. They either " .
-      "have a matching Dbxref (e.g. PubMed ID) or match on the unique constraint set by the Tripal publication module ".
-      "configuration page.  \nCitation: %title %dbxref.\nMatching Pub ids: %ids",
-       array(
-         '%num' => count($pub_ids),
-         '%title' => $pub_details['Citation'],
-         '%dbxref' => $pub_details['Publication Dbxref'],
-        '%ids' => implode(",", $pub_ids),
-       )
+    tripal_report_error($message_type, TRIPAL_NOTICE,
+      "The following publication exists %num times on this site:  %title %dbxref (Matching Pub id: %ids). Skipping.",
+       ['%num' => count($pub_ids),
+        '%title' => $pub_details['Citation'],
+        '%dbxref' => $pub_details['Publication Dbxref'],
+        '%ids' => implode(",", $pub_ids)],
+      $message_opts
     );
     $action = 'skipped';
     return FALSE;
@@ -1023,7 +1053,7 @@ function tripal_pub_add_publication($pub_details, &$action, $do_contact = FALSE,
     $pub_id = $pub_ids[0];
   }
 
-  // get the publication type (use the first publication type)
+  // Get the publication type (use the first publication type).
   if (array_key_exists('Publication Type', $pub_details)) {
     $pub_type = '';
     if(is_array($pub_details['Publication Type'])) {
@@ -1041,19 +1071,21 @@ function tripal_pub_add_publication($pub_details, &$action, $do_contact = FALSE,
     $pub_type = chado_get_cvterm($identifiers);
   }
   else {
-    tripal_report_error('tripal_pub', TRIPAL_ERROR,
-      "The Publication Type is a required property but is missing", array());
+    tripal_report_error($message_type, TRIPAL_ERROR,
+      "The Publication Type is a required property but is missing", [], $error_opts);
     $action = 'error';
     return FALSE;
   }
   if (!$pub_type) {
-    tripal_report_error('tripal_pub', TRIPAL_ERROR, "Cannot find publication type: '%type'",
-      array('%type' => $pub_details['Publication Type'][0]));
+    tripal_report_error($message_type, TRIPAL_ERROR, 
+      "Cannot find publication type: '%type'",
+      ['%type' => $pub_details['Publication Type'][0]], $error_opts);
     $action = 'error';
     return FALSE;
   }
 
-  // the series name field in the pub table is only 255 characters, so we should trim just in case
+  // The series name field in the pub table is only 255 characters, so we 
+  // should trim just in case.
   $series_name = '';
   if (array_key_exists('Series_Name', $pub_details)) {
     $series_name = substr($pub_details['Series Name'], 0, 255);
@@ -1062,7 +1094,7 @@ function tripal_pub_add_publication($pub_details, &$action, $do_contact = FALSE,
     $series_name = substr($pub_details['Journal Name'], 0, 255);
   }
 
-  // build the values array for inserting or updating
+  // Build the values array for inserting or updating.
   $values = array(
     'title'       => $pub_details['Title'],
     'volume'      => (isset($pub_details['Volume'])) ? $pub_details['Volume'] : '',
@@ -1074,13 +1106,14 @@ function tripal_pub_add_publication($pub_details, &$action, $do_contact = FALSE,
     'type_id'     => $pub_type->cvterm_id,
   );
 
-  // if there is no pub_id then we need to do an insert.
+  // If there is no pub_id then we need to do an insert.
   if (!$pub_id) {
     $options = array('statement_name' => 'ins_pub_tivoseispypaunty');
     $pub = chado_insert_record('pub', $values, $options);
     if (!$pub) {
-      tripal_report_error('tripal_pub', TRIPAL_ERROR, "Cannot insert the publication with title: %title",
-      array('%title' => $pub_details['Title']));
+      tripal_report_error($message_type, TRIPAL_ERROR, 
+        "Cannot insert the publication with title: %title",
+        ['%title' => $pub_details['Title']], $error_opts);
       $action = 'error';
       return FALSE;
     }
@@ -1088,22 +1121,24 @@ function tripal_pub_add_publication($pub_details, &$action, $do_contact = FALSE,
     $action = 'inserted';
   }
 
-  // if there is a pub_id and we've been told to update, then do the update
-  if ($pub_id and $update_if_exists) {
+  // If there is a pub_id and we've been told to update, then do the update.
+  else if ($pub_id and $update_if_exists) {
     $match = array('pub_id' => $pub_id);
     $options = array('statement_name' => 'up_pub_tivoseispypaunty');
     $success = chado_update_record('pub', $match, $values, $options);
     if (!$success) {
-      tripal_report_error('tripal_pub', TRIPAL_ERROR, "Cannot update the publication with title: %title",
-      array('%title' => $pub_details['Title']));
+      tripal_report_error($message_type, TRIPAL_ERROR, 
+        "Cannot update the publication with title: %title",
+        ['%title' => $pub_details['Title']], $error_opts);
       $action = 'error';
       return FALSE;
     }
     $action = 'updated';
   }
 
-  // before we add any new properties we need to remove those that are there if this
-  // is an update.  The only thing we don't want to remove are the 'Publication Dbxref'
+  // Before we add any new properties we need to remove those that are there 
+  // if this is an update.  The only thing we don't want to remove are the 
+  // 'Publication Dbxref'.
   if ($update_if_exists) {
     $sql = "
       DELETE FROM {pubprop}
@@ -1118,14 +1153,22 @@ function tripal_pub_add_publication($pub_details, &$action, $do_contact = FALSE,
     chado_query($sql, array(':pub_id' => $pub_id));
   }
 
-  // iterate through the properties and add them
+  // Iterate through the properties and add them.
   foreach ($pub_details as $key => $value) {
-    // the pub_details may have the raw search data (e.g. in XML from PubMed.  We'll irgnore this for now
+    
+    // The pub_details may have the raw search data (e.g. in XML from PubMed. 
+    // We'll irgnore this for now.
     if($key == 'raw') {
       continue;
     }
+    
+    // Since we're not updating the 'Publication Dbxref' on an update
+    // skip this property.
+    if ($update_if_exists and $key == 'Publication Dbxref') {
+      continue;
+    }
 
-    // get the cvterm by name
+    // Get the cvterm by name.
     $identifiers = array(
       'name' => $key,
       'cv_id' => array(
@@ -1134,8 +1177,7 @@ function tripal_pub_add_publication($pub_details, &$action, $do_contact = FALSE,
     );
     $cvterm = chado_get_cvterm($identifiers);
 
-    // if we could not find the cvterm by name then try by synonym
-    //$cvterm = chado_get_cvterm(array('name' => $key, 'cv_id' => array('name' => 'tripal_pub')));
+    // If we could not find the cvterm by name then try by synonym.
     if (!$cvterm) {
       $identifiers = array(
         'synonym' => array(
@@ -1146,11 +1188,12 @@ function tripal_pub_add_publication($pub_details, &$action, $do_contact = FALSE,
       $cvterm = chado_get_cvterm($identifiers);
     }
     if (!$cvterm) {
-      tripal_report_error('tripal_pub', TRIPAL_ERROR, "Cannot find term: '%prop'. Skipping.", array('%prop' => $key));
+      tripal_report_error($message_type, TRIPAL_ERROR, 
+        "Cannot find term: '%prop'. Skipping.", ['%prop' => $key], $error_opts);
       continue;
     }
 
-    // skip details that won't be stored as properties
+    // Skip details that won't be stored as properties.
     if ($key == 'Author List') {
       tripal_pub_add_authors($pub_id, $value, $do_contact);
       continue;
@@ -1163,8 +1206,9 @@ function tripal_pub_add_publication($pub_details, &$action, $do_contact = FALSE,
     $success = 0;
     if (is_array($value)) {
       foreach ($value as $subkey => $subvalue) {
-        // if the key is an integer then this array is a simple list and
-        // we will insert using the primary key. Otheriwse, use the new key
+        
+        // If the key is an integer then this array is a simple list and
+        // we will insert using the primary key. Otheriwse, use the new key.
         if(is_int($subkey)) {
           $success = chado_insert_property(
             array('table' => 'pub', 'id' => $pub_id),
@@ -1187,8 +1231,9 @@ function tripal_pub_add_publication($pub_details, &$action, $do_contact = FALSE,
       );
     }
     if (!$success) {
-      tripal_report_error('tripal_pub', TRIPAL_ERROR, "Cannot add property '%prop' to publication. Skipping.",
-      array('%prop' => $key));
+      tripal_report_error($message_type, TRIPAL_ERROR, 
+        "Cannot add property '%prop' to pubprop table. Skipping.",
+        ['%prop' => $key], $error_opts);
       continue;
     }
   }
@@ -1483,85 +1528,7 @@ function tripal_pub_get_publication_array($pub_id, $skip_existing = TRUE) {
 
 
 
-/**
- * Imports all publications for a given publication import setup.
- *
- * @param $import_id
- *   The ID of the import setup to use
- * @param $job_id
- *   The jobs management job_id for the job if this function is run as a job.
- *
- * @ingroup tripal_pub
- */
-function tripal_execute_pub_importer($import_id, $job_id = NULL) {
-  print "\nNOTE: Loading of publications is performed using a database transaction. \n" .
-      "If the load fails or is terminated prematurely then the entire set of \n" .
-      "insertions/updates is rolled back and will not be found in the database\n\n";
-
-  // start the transaction
-  $transaction = db_transaction();
-
-  try {
-    $page = 0;
-    $do_contact = FALSE;
-    $num_to_retrieve = 100;
-
-    // get all of the loaders
-    $args = array(':import_id' => $import_id);
-    $sql = "SELECT * FROM {tripal_pub_import} WHERE pub_import_id = :import_id ";
-    $import = db_query($sql, $args)->fetchObject();
-
-    print "Executing Importer: '" . $import->name . "'\n";
-
-    $criteria = unserialize($import->criteria);
-    $remote_db = $criteria['remote_db'];
-    $total_pubs = 0;
-    do {
-      // retrieve the pubs for this page. We'll retreive 100 at a time
-      $results  = tripal_get_remote_pubs($remote_db, $criteria, $num_to_retrieve, $page);
-      $pubs     = $results['pubs'];
-      $num_pubs = $rseults['total_records'];
-      $total_pubs += $num_pubs;
-      tripal_pub_add_publications($pubs, $import->do_contact);
-      $page++;
-    }
-    // continue looping until we have a $pubs array that does not have
-    // our requested numer of records.  This means we've hit the end
-    while (count($pubs) == $num_to_retrieve);
-
-    // For backwards compatibility check to see if the legacy pub module
-    // is enabled. If so, then sync the nodes.
-    if (module_exists('tripal_pub')) {
-      // sync the newly added publications with Drupal. If the user
-      // requested a report then we don't want to print any syncing information
-      // so pass 'FALSE' to the sync call
-      print "Syncing publications with Drupal...\n";
-      chado_node_sync_records('pub');
-
-      // if any of the importers wanted to create contacts from the authors then sync them
-      if($import->do_contact) {
-        print "Syncing contacts with Drupal...\n";
-        chado_node_sync_records('contact');
-      }
-    }
-    // For Tripal v3 we want to publish the new publications
-    else {
-      $bundle = tripal_load_bundle_entity(['label' => 'Publication']);
-      if ($bundle) {
-        chado_publish_records(['bundle_name' => $bundle->name], $job_id);
-      }
-    }
-    tripal_set_job_progress($job_id, '100');
-  }
-  catch (Exception $e) {
-    $transaction->rollback();
-    print "\n"; // make sure we start errors on new line
-    watchdog_exception('T_pub_import', $e);
-    print "FAILED: Rolling back database changes...\n";
-    return;
-  }
-  print "Done.\n";
-}
+
 
 /**
  * This function is used to perfom a query using one of the supported databases

tripal_chado/includes/tripal_chado.bundle.inc

@@ -56,6 +56,9 @@ function tripal_chado_bundle_create($bundle, $storage_args) {
     if (array_key_exists('type_value', $storage_args)) {
       $record['type_value'] = $storage_args['type_value'];
     }
+    if (array_key_exists('base_type_id', $storage_args)) {
+      $record['base_type_id'] = $storage_args['base_type_id'];
+    }
     $success = drupal_write_record('chado_bundle', $record);
     if (!$success) {
       throw new Exception('Cannot create content type. Problem associating type with Chado.');

tripal_chado/includes/tripal_chado.entity.inc

@@ -23,6 +23,9 @@ function tripal_chado_entity_create(&$entity, $type, $bundle = NULL) {
       $entity->chado_table =  NULL;
       $entity->chado_column = NULL;
       $entity->chado_linker = NULL;
+      $entity->chado_type_id = NULL;
+      $entity->chado_type_value = NULL;
+      $entity->chado_base_type_id = NULL;
 
       // Add in the Chado table information for this entity type.
       if (!$bundle) {
@@ -32,6 +35,9 @@ function tripal_chado_entity_create(&$entity, $type, $bundle = NULL) {
         $entity->chado_table = $bundle->data_table;
         $entity->chado_column = $bundle->type_column;
         $entity->chado_linker = $bundle->type_linker_table;
+        $entity->chado_type_id = $bundle->type_id;
+        $entity->chado_type_value = $bundle->type_value;
+        $entity->chado_base_type_id = $bundle->base_type_id;
       }
     }
     if (!property_exists($entity, 'chado_record')) {
@@ -74,6 +80,7 @@ function tripal_chado_entity_load($entities, $type) {
           $bundle->type_column = $chado_bundle->type_column;
           $bundle->type_id = $chado_bundle->type_id;
           $bundle->type_value = $chado_bundle->type_value;
+          $bundle->base_type_id = $chado_bundle->base_type_id;
         }
       }
     }
@@ -94,8 +101,6 @@ function tripal_chado_entity_load($entities, $type) {
         if (!$bundle) {
           continue;
         }
-        // TODO: this may need fixing. The chado_column may not always
-        // be the type_id of the base table. Is it expected to be so here?
         $entity->chado_table = $bundle->data_table;
         $entity->chado_column = $bundle->type_column;
 

tripal_chado/includes/tripal_chado.fields.inc

@@ -19,8 +19,10 @@ function tripal_chado_bundle_fields_info($entity_type, $bundle) {
     'chado_cvterm_id' => $chado_bundle->type_id,
     'chado_table' => $chado_bundle->data_table,
     'chado_type_table' => $chado_bundle->type_linker_table,
+    'chado_type_id' => $chado_bundle->type_id,
     'chado_type_column' => $chado_bundle->type_column,
     'chado_type_value' => $chado_bundle->type_value,
+    'chado_base_type_id' => $chado_bundle->base_type_id,
   );
 
   $info = array();
@@ -220,13 +222,16 @@ function tripal_chado_bundle_fields_info_custom(&$info, $details, $entity_type,
   $type_column = $details['chado_type_column'];
   $cvterm_id  = $details['chado_cvterm_id'];
   $type_value = $details['chado_type_value'];
+  $base_type_id = $details['chado_base_type_id'];
 
   $schema = chado_get_schema($table_name);
 
-  // Handle type_id fields that are not the type_column, except for the
-  // organism type_id as that is handled by the taxrank__infraspecific_taxon
-  // field.
-  if (array_key_exists('type_id', $schema['fields']) and 'type_id' != $type_column and $table_name != 'organism') {
+  // Add the additional_type field to all tables with a type_id that is not used
+  // as the type column nor has a $base_type_id (i.e. the content type uses a 
+  // prop or linker table to resolve the type).
+  if (array_key_exists('type_id', $schema['fields']) and 'type_id' != $type_column and 
+      !$base_type_id and $table_name != 'organism') {    
+
     $field_name = 'schema__additional_type';
     $field_type = 'schema__additional_type';
     $info[$field_name] = array(
@@ -713,7 +718,7 @@ function tripal_chado_bundle_fields_info_linker(&$info, $details, $entity_type,
   $prop_table = $table_name . 'prop';
   if (chado_table_exists($prop_table)) {
     
-    $props = tripal_chado_bundle_get_properties($table_name, $prop_table, $type_table, $type_column);
+    $props = tripal_chado_bundle_get_properties($table_name, $prop_table, $type_table, $type_column, $cvterm_id);
     foreach ($props as $term) {
           
       $field_name = strtolower(preg_replace('/[^\w]/','_', $term->dbxref_id->db_id->name . '__' . $term->name));
@@ -855,8 +860,10 @@ function tripal_chado_bundle_instances_info($entity_type, $bundle) {
     'chado_cvterm_id' => $chado_bundle->type_id,
     'chado_table' => $chado_bundle->data_table,
     'chado_type_table' => $chado_bundle->type_linker_table,
+    'chado_type_id' => $chado_bundle->type_id,
     'chado_type_column' => $chado_bundle->type_column,
     'chado_type_value' => $chado_bundle->type_value,
+    'chado_base_type_id' => $chado_bundle->base_type_id,
   );
 
   $info = array();
@@ -1280,10 +1287,14 @@ function tripal_chado_bundle_instances_info_custom(&$info, $entity_type, $bundle
   $type_column = $details['chado_type_column'];
   $cvterm_id  = $details['chado_cvterm_id'];
   $type_value = $details['chado_type_value'];
+  $base_type_id = $details['chado_base_type_id'];  
   $schema = chado_get_schema($table_name);
 
-  // An additional type for publications
-  if (array_key_exists('type_id', $schema['fields']) and 'type_id' != $type_column and $table_name != 'organism') {
+  // Add the additional_type field to all tables with a type_id that is not used
+  // as the type column nor has a $base_type_id (i.e. the content type uses a
+  // prop or linker table to resolve the type).
+  if (array_key_exists('type_id', $schema['fields']) and 'type_id' != $type_column and
+      !$base_type_id and $table_name != 'organism') { 
     $field_name = 'schema__additional_type';
     $is_required = FALSE;
     if (array_key_exists('not null', $schema['fields']['type_id']) and
@@ -2478,7 +2489,7 @@ function tripal_chado_bundle_instances_info_linker(&$info, $entity_type, $bundle
   $prop_table = $table_name . 'prop';
   if (chado_table_exists($prop_table)) {
 
-    $props = tripal_chado_bundle_get_properties($table_name, $prop_table, $type_table, $type_column);
+    $props = tripal_chado_bundle_get_properties($table_name, $prop_table, $type_table, $type_column, $cvterm_id);
     foreach ($props as $term) {     
 
        $field_name = strtolower(preg_replace('/[^\w]/','_', $term->dbxref_id->db_id->name . '__' . $term->name));
@@ -2750,7 +2761,7 @@ function tripal_chado_bundle_instances_info_linker(&$info, $entity_type, $bundle
  * @return
  *   An array of cvterm objects for the properties to be added as fields.
  */
-function tripal_chado_bundle_get_properties($table_name, $prop_table, $type_table, $type_column) {
+function tripal_chado_bundle_get_properties($table_name, $prop_table, $type_table, $type_column, $cvterm_id) {
   
   $tschema = chado_get_schema($table_name);
   $schema = chado_get_schema($prop_table);
@@ -2817,6 +2828,9 @@ function tripal_chado_bundle_get_properties($table_name, $prop_table, $type_tabl
   // Iterate through all of the properties and do some final checks to see
   // which ones should be added.
   $prop_arr = [];
+  if (!$props) {
+    return $prop_arr;
+  }
   while ($prop = $props->fetchObject()) {
     $term = chado_generate_var('cvterm', array('cvterm_id' => $prop->type_id));
     $term = chado_expand_var($term, 'field', 'cvterm.definition');

tripal_chado/includes/tripal_chado.vocab_storage.inc

@@ -368,4 +368,4 @@ function tripal_chado_vocab_import_form_validate($form, &$form_state) {
 function tripal_chado_vocab_import_form_submit($form, &$form_state) {
   module_load_include('inc', 'tripal_chado', 'includes/loaders/tripal_chado.obo_loader');
   return tripal_cv_obo_form_submit($form, $form_state);
-}
+}

tripal_chado/tripal_chado.drush.inc

@@ -84,36 +84,42 @@ function tripal_chado_drush_command() {
     ),
   );
 
+  //
   // Drush commands for publications
+  //
   $items['trp-import-pubs'] = array(
     'description' => dt('Imports publications from remote databases using saved configuration settings.'),
     'options' => array(
-      'create_contacts' => dt('provide this option to create or update contacts for authors. By default contacts are not created or updated.'),
+      'create_contacts' => dt('Create or update contacts for authors. This option is only available with the --dxref option because creation of contacts is otherwise set for each importer.'),
       'dbxref' => dt('An accession number for a publication from a remote database (e.g. PMID:23582642).'),
-      'report' => dt("Set to the email address of the recipient who should receive an HTML report of the publications that have been added."),
+      'report' => dt("Set to the email address of the recipient who should receive an HTML report of the publications that have been added. Not applicable with the --dbxref option."),
       'update' => dt("Set to 'Y' to update existing pubs.  By default only new pubs are inserted."),
+      'publish' => dt("Set to 'Y' to publish publications after import into Chado. (default: Y)."),
+      'sync' => dt("For Tripal v3 legacy mode only. Set to 'Y' to sync publications after import into Chado. This will create legacy \"nodes\" rather than the newer Tripal v3 entities."),
       'username' => array(
         'description' => dt('The Drupal user name for which the job should be run.  The permissions for this user will be used.'),
       ),
     ),
     'examples' => array(
-      'Standard example' => 'drush tripal-pubs-import',
-      'Standard example' => 'drush -l http://[site url] tripal-pubs-import --report=[email]. Where [site url] is the URL of the website and [email] is the email address of the recipient to receive the HTML report',
-      'Import single publication' => 'drush tripal-pub-import --dbxref=PMID:23582642',
+      'Standard example' => 'drush trp-import-pubs --username=[username] --report=[email]. Where [site url] is the URL of the website and [email] is the email address of the recipient to receive the HTML report',
+      'Import single publication' => 'drush tripal-pub-import --username=[username] --dbxref=PMID:23582642',
     ),
   );
   $items['trp-update-pubs'] = array(
     'description' => dt('Updates publication information for publications with a supported database cross-reference.'),
     'options' => array(
-      'create_contacts' => dt('provide this option to create or update contacts for authors. By default contacts are not created or updated.'),
       'dbxref' => dt('An accession number for a publication from a remote database (e.g. PMID:23582642)'),
       'db' => dt('The database name (e.g. PMID or AGL)'),
+      'publish' => dt("Set to 'Y' to publish publications after import into Chado. (default: Y)."),
+      'sync' => dt("For Tripal v3 legacy mode only. Set to 'Y' to sync publications after import into Chado. This will create legacy \"nodes\" rather than the newer Tripal v3 entities."),
+      'username' => array(
+        'description' => dt('The Drupal user name for which the job should be run.  The permissions for this user will be used.'),
+      ),
     ),
     'examples' => array(
-      'Standard example' => 'drush tripal-pubs-update',
-      'Create contacts during update' => 'drush tripal-pubs-update --create_contacts=1',
-      'Update a single record' => 'drush tripal-pubs-update --dbxref=PMID:23582642',
-      'Update all records for a single database' => 'drush tripal-pubs-update --db=PMID'
+      'Standard example' => 'drush trp-update-pubs --username=[username]',
+      'Update a single record' => 'drush trp-update-pubs --username=[username] --dbxref=PMID:23582642',
+      'Update all records for a single database' => 'drush trp-update-pubs --username=[username] --db=PMID'
     ),
   );
 
@@ -126,9 +132,14 @@ function tripal_chado_drush_command() {
  * @ingroup tripal_drush
  */
 function drush_tripal_chado_trp_import_pubs() {
+  // The create contacts only goes with the dbxref argument
   $create_contacts = drush_get_option('create_contacts');
   $dbxref = drush_get_option('dbxref');
+  
+  // The do_report, publish, sync only go with the normal importer.
   $do_report = drush_get_option('report');
+  $publish = drush_get_option('publish');
+  $sync = drush_get_option('sync');
   $update = drush_get_option('update');
   $uname = drush_get_option('username');
 
@@ -140,13 +151,27 @@ function drush_tripal_chado_trp_import_pubs() {
   else {
     $update = FALSE;
   }
+  
+  
+  if ($publish == 'Y' and $sync == 'Y' ) {
+    $publish = 'both';
+  }
+  elseif ($publish != 'Y' and $sync == 'Y') {
+    $publish = 'sync';
+  }
+  elseif ($publish == 'Y' and $sync != 'Y') {
+    $publish = TRUE;
+  }
+  else {
+    $publish = FALSE;
+  }
 
   module_load_include('inc', 'tripal_chado', 'includes/loaders/tripal_chado.pub_importers');
   if ($dbxref) {
-    chado_import_pub_by_dbxref($dbxref, $create_contacts, $update);
+    chado_import_pub_by_dbxref($dbxref, $create_contacts, $publish, $update);
   }
   else {
-    chado_execute_active_pub_importers($do_report, $update);
+    chado_execute_active_pub_importers($do_report, $publish, $update);
   }
 }
 
@@ -159,9 +184,27 @@ function drush_tripal_chado_trp_update_pubs() {
   $create_contacts = drush_get_option('create_contacts');
   $dbxref = drush_get_option('dbxref');
   $db = drush_get_option('db');
+  $publish = drush_get_option('publish');
+  $sync = drush_get_option('sync');
+  
+  $uname = drush_get_option('username');
+  drush_tripal_set_user($uname);
+  
+  if ($publish == 'Y' and $sync == 'Y' ) {
+    $publish = 'both';
+  }
+  elseif ($publish  != 'Y' and $sync == 'Y' ) {
+    $publish = 'sync';
+  }
+  elseif ($publish  == 'Y'  and $sync  != 'Y' ) {
+    $publish = TRUE;
+  }
+  else {
+    $publish = FALSE;
+  }
 
   module_load_include('inc', 'tripal_chado', 'includes/loaders/tripal_chado.pub_importers');
-  chado_reimport_publications($create_contacts, $dbxref, $db);
+  chado_reimport_publications($create_contacts, $dbxref, $db, $publish);
 }
 
 /**

tripal_chado/tripal_chado.install

@@ -619,6 +619,11 @@ function tripal_chado_chado_bundle_schema() {
         'type' => 'text',
         'not null' => FALSE,
         'default' => '',
+      ),
+      'base_type_id' => array(
+        'description' => 'If a property table is used for a linker, and if the base table requires a type_id then this is the type that should be used on insert of new records in the base table.',
+        'size' => 'big',        
+        'type' => 'int',
       )
     ),
     'indexes' => array(
@@ -1595,4 +1600,23 @@ function tripal_chado_update_7329() {
   }
 }
 
+# tripal_chado_update_7330 is in PR #596 and adds categories for content types.
 
+/**
+ * Adds a base_type_id to the chado_bundle table.
+ */
+function tripal_chado_update_7331() {
+  try {
+    if (!db_field_exists('chado_bundle', 'base_type_id')) {
+      db_add_field('chado_bundle', 'base_type_id', [
+        'description' => 'If a property table is used for a linker, and if the base table requires a type_id then this is the type that should be used on insert of new records in the base table.',
+        'size' => 'big',
+        'type' => 'int',
+      ]);
+    }
+  }
+  catch (\PDOException $e) {
+    $error = $e->getMessage();
+    throw new DrupalUpdateException('Could not perform update: '. $error);
+  }
+}

tripal_chado/tripal_chado.module

@@ -1190,6 +1190,66 @@ function tripal_feature_match_features_page($id) {
     ));
     return $output;
   }
+}
 
-
+/**
+ * Implements hook_mail().
+ *
+ * @ingroup tripal_legacy_pub
+ */
+function tripal_chado_mail($key, &$message, $params) {
+  $site_name = variable_get('site_name');
+  $language = $message['language'];
+  switch($key) {
+    case 'import_report':
+      
+      $content = [];
+      $content[] = [
+        '#type' => 'markup',
+        '#markup' => '<h3>Tripal Bulk Publication Report</h3>'
+      ];
+      $report = $params['report'];
+
+      foreach ($report as $action => $pubs) {
+        if (count($pubs) > 0) {
+          $title = count($pubs) . ' publication(s) were ' . $action . ":";
+          if ($action == 'error') {
+            $title = count($pubs) . ' publications were not imported due to errors:';
+          }
+          $content[] = [
+            '#type' => 'item',
+            '#title' => $title, 
+            '#markup' => theme_item_list([
+              'title' => '',
+              'type' => 'ol', 
+              'items' => $pubs,
+              'attributes' => [],
+            ]),
+          ];
+        }
+      }
+      $content = '<html>' . drupal_render($content) . '</html';
+      
+      $message['subject'] = t('Publication import from !site', array('!site' => $site_name));
+      
+      
+      if (module_exists('htmlmail') or module_exists('mimemail')) {
+        $headers = array(
+          'MIME-Version' => '1.0',
+          'Content-Type' => 'text/html; charset=UTF-8; format=flowed',
+          'Content-Transfer-Encoding' => '8Bit',
+          'X-Mailer' => 'Drupal'
+        );
+        foreach ($headers as $key => $value) {
+          $message['headers'][$key] = $value;
+        }
+        $message['body'][] = $content;
+      }
+      else {
+        $message['body'][] =  drupal_html_to_text($content);
+      }
+            
+      break;
+  }
 }
+