Merge branch '6.x-3.x' into tv3-cvterm-entities-wont-sync

CONTRIBUTING.md

@@ -1 +1 @@
-Please see the [Tripal online documentation](https://tripal.readthedocs.io/en/latest/dev_guide/contributing.html) for more information about contributing to Tripal and Tripal governance.
+Please see the [Tripal online documentation](https://tripal.readthedocs.io/en/latest/contributing/pull_requests.html#pull-request-pr-guideline) for more information about contributing to Tripal and Tripal governance.

docs/contributing/pull_requests.rst

@@ -61,11 +61,11 @@ Pull Request (PR) Guideline
 The goal of this document is to make it easy for **A)** contributors to make pull requests that will be accepted, and **B)** Tripal committers to determine if a pull request should be accepted.
 
 - PRs that address a specific issue **must** link to the related issue page.
-    - Really in almost every case, there should be an issue for a PR.  This allows feedback and discussion before the coding happens.  Not grounds to reject, but encourage users to create issues at start of their PR.  Better late than never :).
-- Each PR **must** be tested/approved by at least 2 users with at least one user being a "trusted committer."
-    - Testers **should** describe how the testing was performed if applicable (allows others to replicate the test).
-    - At the Project Management Committee's (PMC) discretion, a PR may be subject to only one review.  Generally these are small and obvious commits.
-    - While Tripal's review body is small, only one of the code reviews must be a thorough functional test.
+    - In almost every case, there should be an issue for a PR.  This allows feedback and discussion before the coding happens.  Not grounds to reject, but encourage users to create issues at start of their PR.  Better late than never :).
+- Each PR **must** be tested/approved by at least 1 contributor, if approved, a "trusted committer" will merge the PR.
+    - Testers **should** describe how the testing was performed if applicable (allows others to replicate the test).    
+    - While Tripal's review body is small, the code review must be a thorough functional test.
+    - At the Project Management Committee's (PMC) discretion, a PR may be subject to a non-functional review.  Generally these are small and obvious commits.
     - Tripal's guiding philosophy is to encourage open contribution.  With this in mind, committers should **work with contributors** to resolve issues in their PRs.  PRs that will not be merged should be closed, **transparently citing** the reason for closure.  In an ideal world, features that would be closed are discouraged at the **issue phase** before the code is written!
     - The pull request branch should be deleted after merging (if not from a forked repository) by the person who performs the merge.
 - PRs that include new functionality **must** also provide Unit Tests.

docs/extensions/data_input.rst

@@ -38,9 +38,9 @@ A BibTEX importer for Tripal Publications. Currently this module only provides a
 Tripal Plant PopGen Submission
 -------------------------------
 
-.. image:: https://tripal.readthedocs.io/en/7.x-3.x/_images/Tripal-Bronze.png
-  :target: https://tripal.readthedocs.io/en/7.x-3.x/extensions/module_rating.html#Bronze
-  :alt: Tripal Rating: Bronze
+.. image:: https://tripal.readthedocs.io/en/7.x-3.x/_images/Tripal-Silver.png
+  :target: https://tripal.readthedocs.io/en/7.x-3.x/extensions/module_rating.html#Silver
+  :alt: Tripal Rating: Silver
 
 The Tripal Plant PopGen Submit (TPPS) Module supports a flexible submission interface for genotype, phenotype, environmental, and metadata for population, association, or landscape genetics studies. The portal walks the user through specific questions and collects georeferenced coordinates on plant accessions and also supports ontology standards, including the Minimal Information About a Plant Phenotyping Experiment (MIAPPE) (http://www.miappe.org/) and standard genotyping file formats, such as VCF.
 

docs/extensions/search.rst

@@ -6,7 +6,7 @@ CartograTree
 
 CartograTree is a web-based application that allows researchers to identify, filter, compare, and visualize geo-referenced biotic and abiotic data. Its goal is to support numerous multi-disciplinary research endeavors including: phylogenetics, population structure, and association studies.
 
-`Documentation <https://gitlab.com/TreeGenes/CartograTree/blob/master/README.md>`__
+`Documentation <https://cartogratree.readthedocs.io/en/latest/index.html>`__
 `Repository <https://gitlab.com/TreeGenes/CartograTree>`__
 
 Mainlab Chado Search

docs/user_guide/bulk_loader.rst

@@ -5,7 +5,7 @@ Bulk Loader
 
   Remember you must set the ``$DRUPAL_HOME`` environment variable if you want to cut-and-paste the commands below. See :doc:`./install_tripal/drupal_home`
 
-The bulk loader is a tool that Tripal provides for loading of data contained in tab delimited files. Tripal supports loading of files in standard formats (e.g. ``FASTA``, ``GFF``, ``OBO``), but Chado can support a variety of different biological data types and there are often no community standard file formats for loading these data. For example, there is no file format for importing genotype and phenotype data. Those data can be stored in the feature, stock and natural diversity tables of Chado. The Bulk Loader was introduced in Tripal v1.1 and provides a web interface for building custom data loader. In short, the site developer creates the bulk loader "template". This template can then be used and re-used for any tab delimited file that follows the format described by the template. Additionally, bulk loading templates can be exported allowing Tripal sites to share loaders with one another.  Loading templates that have been shared are available on the Tripal website here: http://tripal.info/extensions/bulk-loader-templates.
+The bulk loader is a tool that Tripal provides for loading of data contained in tab delimited files. Tripal supports loading of files in standard formats (e.g. ``FASTA``, ``GFF``, ``OBO``), but Chado can support a variety of different biological data types and there are often no community standard file formats for loading these data. For example, there is no file format for importing genotype and phenotype data. Those data can be stored in the feature, stock and natural diversity tables of Chado. The Bulk Loader was introduced in Tripal v1.1 and provides a web interface for building custom data loader. In short, the site developer creates the bulk loader "template". This template can then be used and re-used for any tab delimited file that follows the format described by the template. Additionally, bulk loading templates can be exported allowing Tripal sites to share loaders with one another.  
 
 The following commands can be executed to install the Tripal Bulk Loader using Drush:
 

docs/user_guide/web_services.rst

@@ -53,7 +53,7 @@ Structure of a Web Service Response
 
 The initial response in JSON is in the `JSON-LD format <https://json-ld.org/>`_ where the LD means Linked Data.   For example:
 
-.. code-block:: JSON
+.. code-block:: json
 
     {
     "@context": {,
@@ -80,7 +80,7 @@ A notable component of JSON-LD is the **@context** sub array.  It is within this
 
 There are two other special keys used in the JSON-LD response.  These are the **@id** and **@type** keys.  The **@id** indicates the unique URL for this resource and becomes a unique name for the resource.  In the example screenshot above, the **@id** of the initial page of web services is ``http://localhost/web-services``.  This URL will always refer to the initial page for Tripal web services on the given site.  The **@type** identifier specifies what type of resource this URL provides.  In this example, the type is **EntryPoint**.  If a client program is unsure as to what an **EntryPoint** is, then that information is provided in the @context section.  The following line indicates that the term **EntryPoint** expands to the vocabulary term:  vocab:EntryPoint
 
-.. code-block:: JSON
+.. code-block:: json
 
   "EntryPoint": "vocab:EntryPoint",
 
@@ -93,7 +93,7 @@ Primary Services
 
 By default, the only resource that Tripal provides at the initial response level is the content resource.  Any resource at this level is hereafter referred to as primary service.  Tripal is design to allow new web-services to be added to it.  These will be more formally described in the Tripal v3 Developer's Handbook.  In short, a primary service provides a variety of data and services for related content and function.   Each primary resource has a version number to help ensure backwards compatibility as new web services are developed and updated.  For example, the default content service currently has a version of v0.1:
 
-.. code-block:: JSON
+.. code-block:: json
 
   "content": "http://localhost/web-services/content/v0.1"
 
@@ -152,7 +152,7 @@ Currently, Tripal provides the ability to search for content via web services by
 
 Where {name} is the label assigned to the content type (See the Content Type Members section above).   Using this path, clients filter content to a specific content type.  But further refinement is possible.  As a reminder, each member (or entity) on the content type members collection appears similar to the following:
 
-.. code-block:: JSON
+.. code-block:: json
 
   {
     "@id": "http://localhost/web-services/content/v0.1/mRNA/691468",
@@ -164,7 +164,7 @@ Where {name} is the label assigned to the content type (See the Content Type Mem
 
 When retrieving the data for a specific entity something similar to the following (for our mRNA example) may be seen:
 
-.. code-block:: JSON
+.. code-block:: json
 
   "label": "LOC_Os01g01010.1",
   "ItemPage": "http://localhost/bio_data/691468",
@@ -187,7 +187,7 @@ When retrieving the data for a specific entity something similar to the followin
 
 As another reminder, when any of these attributes have a URL then further information about that attribute is obtained by following the URL. In the example below, the relationship term yields results similar to the following:
 
-.. code-block:: JSON
+.. code-block:: json
 
   {
       "@id": "http://localhost/web-services/content/v0.1/mRNA/691468/relationship/0",

tripal_chado/includes/TripalFields/so__genotype/so__genotype.inc

@@ -144,7 +144,7 @@ class so__genotype extends ChadoField {
       ],
     ];
     $record = chado_expand_var($record, 'table', $linker_table, $options);
-    $genotype_linkers = isset($record->$linker_table->$fkey_rcolumn) ? $record->$linker_table->$fkey_rcolumn : '';
+    $genotype_linkers = $record->$linker_table;
     if ($genotype_linkers) {
       foreach ($genotype_linkers as $i => $genotype_linker) {
         $genotype = $genotype_linker->genotype_id;
@@ -166,4 +166,4 @@ class so__genotype extends ChadoField {
     }
   }
 
-}
+}

tripal_chado/includes/TripalImporter/FASTAImporter.inc

@@ -243,7 +243,7 @@ class FASTAImporter extends TripalImporter {
       '#description' => t('Enter the regular expression that will extract the unique
                          name needed to identify the existing sequence for which the
                          relationship type selected above will apply.  If no regular
-                         expression is provided, the parent unique name must be the 
+                         expression is provided, the parent unique name must be the
                          same as the loaded feature name.'),
       '#weight' => 6,
     ];
@@ -319,6 +319,20 @@ class FASTAImporter extends TripalImporter {
       form_set_error('db_id', t("Please select a database"));
     }
 
+    // Check to make sure the regexps are valid.
+    if ($re_name && @preg_match("/$re_name/", null) === false) {
+      form_set_error('re_name', t("please provide a valid regular expression for the feature name."));
+    }
+    if ($re_uname && @preg_match("/$re_uname/", null) === false) {
+      form_set_error('re_uname', t("please provide a valid regular expression for the feature unique name."));
+    }
+    if ($re_accession && @preg_match("/$re_accession/", null) === false) {
+      form_set_error('re_accession', t("please provide a valid regular expression for the external database accession."));
+    }
+    if ($re_subject && @preg_match("/$re_subject/", null) === false) {
+      form_set_error('re_subject', t("please provide a valid regular expression for the relationship parent."));
+    }
+
     // check to make sure the types exists
     $cvtermsql = "
       SELECT CVT.cvterm_id
@@ -498,7 +512,10 @@ class FASTAImporter extends TripalImporter {
     $num_seqs = 0;
     $prev_pos = 0;
     $set_start = FALSE;
+    $i = 0;
+
     while ($line = fgets($fh)) {
+      $i++;
       $num_read += strlen($line);
 
       // If we encounter a definition line then get the name, uniquename,
@@ -651,7 +668,7 @@ class FASTAImporter extends TripalImporter {
       ], $values);
       if (count($results) > 1) {
         $this->logMessage("Multiple features exist with the name '!name' of type '!type' for the organism.  skipping",
-          ['!name' => $name, '!type' => $type], TRIPAL_ERROR);
+          ['!name' => $name, '!type' => $cvterm->name], TRIPAL_ERROR);
         return 0;
       }
       if (count($results) == 1) {
@@ -670,7 +687,7 @@ class FASTAImporter extends TripalImporter {
       $results = chado_select_record('feature', ['feature_id'], $values);
       if (count($results) > 1) {
         $this->logMessage("Multiple features exist with the name '!name' of type '!type' for the organism.  skipping",
-          ['!name' => $name, '!type' => $type], TRIPAL_WARNING);
+          ['!name' => $name, '!type' => $cvterm->name], TRIPAL_WARNING);
         return 0;
       }
       if (count($results) == 1) {
@@ -711,7 +728,7 @@ class FASTAImporter extends TripalImporter {
       if (!$success) {
         $this->logMessage("Failed to insert feature '!name (!uname)'", [
           '!name' => $name,
-          '!uname' => $numane,
+          '!uname' => $uname,
         ], TRIPAL_ERROR);
         return 0;
       }
@@ -730,7 +747,7 @@ class FASTAImporter extends TripalImporter {
       else {
         $this->logMessage("Failed to retreive newly inserted feature '!name (!uname)'", [
           '!name' => $name,
-          '!uname' => $numane,
+          '!uname' => $uname,
         ], TRIPAL_ERRORR);
         return 0;
       }
@@ -772,7 +789,7 @@ class FASTAImporter extends TripalImporter {
               [
                 '!name' => $name,
                 '!uname' => $uname,
-                '!type' => $type,
+                '!type' => $cvterm->name,
               ], TRIPAL_ERROR);
             return 0;
           }
@@ -861,7 +878,7 @@ class FASTAImporter extends TripalImporter {
         }
         else {
           $this->logMessage("Failed to retreive newly inserted dbxref '!name (!uname)'",
-            ['!name' => $name, '!uname' => $numane], TRIPAL_ERROR);
+            ['!name' => $name, '!uname' => $uname], TRIPAL_ERROR);
           return 0;
         }
       }