Initial Reorganiztion of Tv3 docs

docs/dev_guide/custom_field/create_instance.rst

@@ -1,9 +1,9 @@
 Attach Fields to Content Types
 ==============================
-In summary, creation of a new field requires creation of three classes that inherit from the ``TripalField``, ``TripalFieldWidget`` and ``TripalFieldFormatter`` base classes.  If the fields are created correctly and placed in the ``includes/TripalFields`` directory of your module then Tripal will automatically find them.  However, the field is not yet attached to any content type. They must be attached.  Fields can be attached programmatically or via the online Drupal interface by a site admin. 
+In summary, creation of a new field requires creation of three classes that inherit from the ``TripalField``, ``TripalFieldWidget`` and ``TripalFieldFormatter`` base classes.  If the fields are created correctly and placed in the ``includes/TripalFields`` directory of your module then Tripal will automatically find them.  However, the field is not yet attached to any content type. They must be attached.  Fields can be attached programmatically or via the online Drupal interface by a site admin.
 
 The hook_bundle_fields_info() function
--------------------------------------
+--------------------------------------
  The three TripalField classes simply define how the field will function, but Drupal does not yet know about the field.  The ``hook_bundle_fields_info`` function tells Drupal about your field. It must be implemented in a custom Drupal module, and provides an array that tells Drupal about the fields and the classes to use for the field.  Suppose we were creating a field named ``obi__genus`` which displays the Genus for a species and we have a custom module named ``tripal_org2``.  The hook function would be named ``tripal_org2_bundle_fields_info()``:
 
 .. code-block:: php
@@ -11,8 +11,8 @@ The hook_bundle_fields_info() function
 
   function tripal_org2_bundle_fields_info($entity_type, $bundle) {
     $info = [];
-    
-    // Make sure this bundle is an organism (OBI:0100026) then we'll attach our 
+
+    // Make sure this bundle is an organism (OBI:0100026) then we'll attach our
     // field to display the genus.
     $term = tripal_load_term_entity(array('term_id' => $bundle->term_id));
     $term_accession = $term->vocab->vocabulary . '__' . $term->accession;
@@ -30,11 +30,11 @@ The hook_bundle_fields_info() function
         'settings' => [],
       ];
    }
-    
+
     return $info
   }
-  
-This function receives as its second argument the ``$bundle`` object. This is the bundle that Drupal is requesting new fields for.  For this example we only want to attach the field if the content type is the organism content type.  The format of the returned ``$info`` array should have the field name as the key and an array that follows the instructions provided by Drupal's `field_create_field() <https://api.drupal.org/api/drupal/modules%21field%21field.crud.inc/function/field_create_field/7.x>`_ function. 
+
+This function receives as its second argument the ``$bundle`` object. This is the bundle that Drupal is requesting new fields for.  For this example we only want to attach the field if the content type is the organism content type.  The format of the returned ``$info`` array should have the field name as the key and an array that follows the instructions provided by Drupal's `field_create_field() <https://api.drupal.org/api/drupal/modules%21field%21field.crud.inc/function/field_create_field/7.x>`_ function.
 
 The settings indicate the field name, the field type, the cardinality (how many values are allowed), any default settings and the storage type.  Because we expect our data to come from Chado we set the ``field_chado_storage`` as the type.  The ``locked`` setting is set to FALSE indicating that Drupal will allow the field to be deleted if the site developer desires.
 
@@ -42,10 +42,10 @@ When the site administrator navigates to **Administer > Structure > Tripal Conte
 
 Programmatically Attaching Fields
 ---------------------------------
-You probably want to programmatically attach fields to content types if your have existing data that you know should be made available. For example, an organism always has a genus and only one genus.  If we have a field that displays the genus for an organism then we will want it automatically attached on installation of our module.  We can do this programmatically using two hook functions: ``hook_bundle_fields_info()`` and ``hook_bundle_instances_info()``.  Both functions are required to attach a field to a content type. 
+You probably want to programmatically attach fields to content types if your have existing data that you know should be made available. For example, an organism always has a genus and only one genus.  If we have a field that displays the genus for an organism then we will want it automatically attached on installation of our module.  We can do this programmatically using two hook functions: ``hook_bundle_fields_info()`` and ``hook_bundle_instances_info()``.  Both functions are required to attach a field to a content type.
 
 The hook_bundle_instances_info() function.
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 The previous hook tells Drupal that our field exists and is allowed to be connected to the organism bundle.  Next we need to create an actual instance of this field for the bundle.  We do this with the ``hook_bundle_instances_info()`` function.  The format is the same as the previous hook but the info array is different.  For example:
 
 .. code-block:: php
@@ -53,13 +53,13 @@ The previous hook tells Drupal that our field exists and is allowed to be connec
 
   function tripal_org2_bundle_instances_info($entity_type, $bundle) {
     $info = []
-    
-    // Make sure this bundle is an organism (OBI:0100026) then we'll attach our 
+
+    // Make sure this bundle is an organism (OBI:0100026) then we'll attach our
     // field to display the genus.
     $term = tripal_load_term_entity(array('term_id' => $bundle->term_id));
     $term_accession = $term->vocab->vocabulary . '__' . $term->accession;
     if ($term_accession == 'OBI:0100026') {
-    
+
       $field_name = 'obi__genus';
       $is_required = FALSE;
       $info[$field_name] =  [
@@ -95,27 +95,27 @@ The previous hook tells Drupal that our field exists and is allowed to be connec
     }
     return $info;
   }
-  
+
 The format of the returned ``$info`` array should have the field name as the key and an array that follows the instructions provided by Drupal's `field_create_instance() <https://api.drupal.org/api/drupal/modules%21field%21field.crud.inc/function/field_create_instance/7.x>`_ function. 
 
 Unique to this info array are the settings related to Chado.  Because we expect our data to be loaded from Chado we must specify these settings:
 
  - ``base_table``: the name of the base table to which the record will be associated. In our case the ``organism`` table of Chado is the base table.
- - ``chado_table``: the name of the actual table form which the value of the field will be loaded or saved to.  In our case the ``organism`` table is also the ``chado_table``.  
+ - ``chado_table``: the name of the actual table form which the value of the field will be loaded or saved to.  In our case the ``organism`` table is also the ``chado_table``.
  - ``chado_column``: the name of the column in the ``chado_table`` where the data is loaded from. if the ``base_table`` and ``chado_table`` are the same then this is the name of the column. In our case the ``genus`` columns.  If the base and chado tables are different then it is the name o the primary key column in the ``chado_table``
- - ``auto_attach``:  set this to TRUE if you want the field to automatically be added to an entity when it is generated for viewing.  Set it to FALSE to allow the field to be added via AJAX. For fields that require time to load setting to FALSE is preferred. 
- 
+ - ``auto_attach``:  set this to TRUE if you want the field to automatically be added to an entity when it is generated for viewing.  Set it to FALSE to allow the field to be added via AJAX. For fields that require time to load setting to FALSE is preferred.
+
 .. note::
   A base table is one that contains the primary records to which ancillary data (e.g. properties, cross references, CV terms, publications, contacts, etc) are associated via linker tables. For example some base tables include: ``feature``, ``organism``, ``stock``, ``library``, etc.).  The ``base_table`` and ``chado_table`` will always be the same when you are mapping a field to data in a column in a base table. If your field maps data to a "linker" table where ancillary data is stored then the ``chado_table`` will be the linker table.
 
 Notice as well that the ``display`` and ``widget`` sections list the name of our TripalEntityWidget and TripalEntityFormatter classes respectively.  This tells drupal to use our widget and formatter classes by default.
 
-When the site administrator navigates to **Administer > Structure > Tripal Content Types**, clicks on a content type, and then the **manage fields** tab, a link appears at the top titled **Check for new fields**.  When that link is clicked, this hook function is called.  
+When the site administrator navigates to **Administer > Structure > Tripal Content Types**, clicks on a content type, and then the **manage fields** tab, a link appears at the top titled **Check for new fields**.  When that link is clicked, this hook function is called.
 
 .. note::
 
   Both hook functions must be properly constructed for the field to be automatically attached to the content type.
-  
+
 Allowing Manual Attachment of Fields
 ------------------------------------
 Not all fields are created equal.  Some fields can be added by the site developer to a bundle and some cannot.  When the ``TripalField`` class is implemented for a class the ``$no_ui`` parameter is set to indicate if a field can be added via the web interface or not.  See the :doc:`manual_field_creation` page for more details. But in short the following setting does not allow a field to be added using the web interface
@@ -123,7 +123,7 @@ Not all fields are created equal.  Some fields can be added by the site develope
 .. code-block::  php
 
  public static $no_ui = TRUE;
- 
+
 The following setting will allow the field to be added:
 
 .. code-block::  php
@@ -136,7 +136,7 @@ The following code is a snippet from the ``tripal_chado_bundle_create_user_field
 
 .. code-block:: php
   :linenos:
-  
+
   function tripal_chado_bundle_create_user_field($new_field, $bundle) {
 
     // Get the table this bundle is mapped to.
@@ -151,20 +151,20 @@ The following code is a snippet from the ``tripal_chado_bundle_create_user_field
     $chado_type_column = $bundle->type_column;
     $chado_type_id = $bundle->type_id;
     $chado_type_value = $bundle->type_value;
-  
+
     // We allow site admins to add new chado_linker__prop fields to an entity.
     // This function will allow us to properly add them.  But at this point we
     // don't know the controlled vocabulary term.  We'll have to use the
     // defaults and let the user set it using the interface.
     if ($new_field['type'] == 'chado_linker__prop') {
       $table_name = $chado_table . 'prop';
-  
+
       if (chado_table_exists($table_name)) {
         $schema = chado_get_schema($table_name);
         $pkey = $schema['primary key'][0];
         $field_name = $new_field['field_name'];
         $field_type = 'chado_linker__prop';
-  
+
         // First add the field.
         field_create_field(array(
           'field_name' => $field_name,
@@ -175,7 +175,7 @@ The following code is a snippet from the ``tripal_chado_bundle_create_user_field
             'type' => 'field_chado_storage',
           ),
         ));
-  
+
         // Now add the instance
         field_create_instance(array(
           'field_name' => $field_name,
@@ -217,7 +217,5 @@ The following code is a snippet from the ``tripal_chado_bundle_create_user_field
 
 
 .. note::
-  
+
   It is possible to have a field that is both programmatically attached to some content types but is also allowed to be attached to another content type by the site admin using the web interface. To do this, programmatically add the field to the content types using the ``hook_bundle_instances_info`` function and also implement the ``hook_bundle_create_user_field`` function to support manual adding.
-  
- 

docs/dev_guide/custom_field/custom_formatter.rst

@@ -1,21 +1,21 @@
 Creating a Custom Formatter
 ===========================
-The third component of a field is the formatter.  Thus far we have introduced how to create a field class and a widget class for a field.  The field class is responsible for describing the field, loading data into it, and providing search support.  The widget class provided a Drupal form for online editing of the field.  Finally, the formatter is responsible for display of the field on a Tripal site.  
- 
+The third component of a field is the formatter.  Thus far we have introduced how to create a field class and a widget class for a field.  The field class is responsible for describing the field, loading data into it, and providing search support.  The widget class provided a Drupal form for online editing of the field.  Finally, the formatter is responsible for display of the field on a Tripal site.
+
 .. note::
-  This guide assumes you already have your formatter class file created. For more information, see :doc:`manual_field_creation` or, :doc:`tripal_field_generator`. 
-  
-The formatter class is the simplest of all the Tripal field classes.  Here we will again use the **obi__organism** field that comes with the ``tripal_chado`` module.  
+  This guide assumes you already have your formatter class file created. For more information, see :doc:`manual_field_creation` or, :doc:`tripal_field_generator`.
+
+The formatter class is the simplest of all the Tripal field classes.  Here we will again use the **obi__organism** field that comes with the ``tripal_chado`` module.
 
 The view() function.
 ~~~~~~~~~~~~~~~~~~~~
-In most cases the only function you need to implement is the ``view()`` function. This function is called whenever your field needs to be displayed on a page. The following code is from the ``obi__organism_formatter.inc`` class file.  
+In most cases the only function you need to implement is the ``view()`` function. This function is called whenever your field needs to be displayed on a page. The following code is from the ``obi__organism_formatter.inc`` class file.
 
 .. code-block:: php
   :linenos:
 
   public function view(&$element, $entity_type, $entity, $langcode, $items, $display) {
-  
+
     if ($items[0]['value']) {
       $content = $items[0]['value']['rdfs:label'];
       if (array_key_exists('entity', $items[0]['value'])) {
@@ -31,19 +31,19 @@ In most cases the only function you need to implement is the ``view()`` function
       );
     }
   }
-  
+
 In the code above the input arguments have the following meaning:
-  - ``$element`` is the first argument. It is an array into which you should set the contents to be displayed.  
+  - ``$element`` is the first argument. It is an array into which you should set the contents to be displayed.
   - ``$entity_type`` will always have the value ``Tripal Entity``.
   - ``$entity`` is the entity object which contains all information about the entity including the loaded data values.
   -  ``$langcode`` is the language. This is used by Drupal to provide translations of data into other spoken languages. By default, Tripal does not use a language, as biological data is generally language agnostic.  Consider for example a gene sequence or a feature coordinate.
-  - ``$items`` is an array containing all of the loaded data for this field.  
+  - ``$items`` is an array containing all of the loaded data for this field.
   - ``$display`` is the name of the display such as full page, a teaser, etc. Currently, Tripal does not distinguish between displays.
-  
-The purpose of the ``view()`` function is to iterate through the values in the ``$items`` array, and format them into an appropriate display for viewing.  Here you must remember the structure of the data in the ``$items`` array.  
- 
+
+The purpose of the ``view()`` function is to iterate through the values in the ``$items`` array, and format them into an appropriate display for viewing.  Here you must remember the structure of the data in the ``$items`` array.
+
 To demonstrate this function, let's look at what we expect in our ``$items`` array. Using the `Citrus sinesis` organism from the User's Guide. We would expect an items array to look like the following:
- 
+
 .. code::
 
   $items = [
@@ -57,53 +57,53 @@ To demonstrate this function, let's look at what we expect in our ``$items`` arr
         "entity" => "TripalEntity:3",
       ],
       "chado-feature__organism_id" => 12,
-    ],    
+    ],
   ];
-  
+
 You may recall that the ``$items`` array structure is the same as that created by the ``load()`` function described in the :doc:`manual_field_creation` page. Note that each key in the ``value`` array is an accession for a controlled vocabulary term.  These accessions are used to unambiguously describe the value. To display the organism on a page we need the element named ``rdfs:label``.  Thus, we set the ``$content`` variable to contain this value as shown on line 4 of the ``view()`` function above.
 
 Because our organisms are also published entities we want to link to their respective pages each time an organism is displayed.  Because the ``value`` array has an element named ``entity`` we know that this item is published.  Lines 5-6 of the ``view()`` function shown above use this information to create a clickable link to the organism page.   Finally, the ``$element`` argument is set to provide content of type ``markup``.  This ``$element`` array is a `Drupal renderable array <https://www.drupal.org/docs/7/api/render-arrays/render-arrays-overview>`_.
 
-Lastly, notice the element named ``chado-feature__organism_id``.  This element is at the same level as the ``value`` element.  This data is meant to be used internally by the field. It maps this fields values to the appropriate table in Chado where the data is stored.  
+Lastly, notice the element named ``chado-feature__organism_id``.  This element is at the same level as the ``value`` element.  This data is meant to be used internally by the field. It maps this fields values to the appropriate table in Chado where the data is stored.
 
-.. warning:: 
+.. warning::
 
   You should never show the user any data that is outside of ``value`` element.  Remember that your field can be shown by other viewers, including web services.  By ensuring that data in the ``value`` element is mean to be displayed we ensure that information on the web page, web services, or any other future form of display is always consistent.
 
 In summary, the following should be observed when processing the ``$items`` array for viewing:
 
-  - A field with only one value (a cardinality of 1) will always have only one element in the ``$items`` array and can use the index 0. This is what has been done in this example code. 
+  - A field with only one value (a cardinality of 1) will always have only one element in the ``$items`` array and can use the index 0. This is what has been done in this example code.
   - A field with more than one value can have any number of elements in the ``$items`` array.  You should therefore iterate through all of them.
   - For every index in ``$item`` you should create a matching index in ``$element`` to display the data found in that ``$item``.
   - If there are no items, then nothing you return will be displayed.
   - For each element in the ``$items`` array there is a ``value`` key.  Only the data in the ``value`` key should be shown to the user.
-  - Each element in the ``$items`` array may have more than a ``value`` key.  These values are meant to help manage the data. 
+  - Each element in the ``$items`` array may have more than a ``value`` key.  These values are meant to help manage the data.
 
 .. warning::
 
   You should never have SQL statements or any API calls that retrieve data in the formatter ``view()`` function. The formatter should strictly format data for viewing.
-  
+
 Creating Pagers
 ~~~~~~~~~~~~~~~
 The example shown in the previous section was for a field that will always only contain a single element.  However some fields may contain a large number of elements.  Consider an mRNA and it's relationships to subfeatures: exons, 5' UTRs, 3'UTRs, CDS, etc.).  A large mRNA can have many relationships.  Alternatively, consider the case where a genetic map content type may have a field that lists all of the markers on the map.  Such a list could become extremely long on the page.  In these cases it may be best to only list a few items at a time and to provide a pager to let the user cycle through the items.  An example of a pager added to the bottom of relationships is shown in the example below.
 
 .. image:: custom_formatter.pager.1.png
 
-To create a pager we first need to calculate the number of items we want to display per page and the total number of pages required to display all of the data.  
+To create a pager we first need to calculate the number of items we want to display per page and the total number of pages required to display all of the data.
 
 .. code-block:: php
-  
+
   $items_per_page = 10;
   $total_records = count($items);
   $total_pages = (int) ($total_records / $items_per_page) + 1;
-  
-Next, we must initialize the pager by calling the ``pager_default_initialize`` function.  We pass it the total number of records, the number of items per page and the index (i.e. ``$pelement``) for this pager on the page.  
+
+Next, we must initialize the pager by calling the ``pager_default_initialize`` function.  We pass it the total number of records, the number of items per page and the index (i.e. ``$pelement``) for this pager on the page.
 
 .. code-block:: php
 
-  $pelement = 0; 
+  $pelement = 0;
   $current_page = pager_default_initialize($total_records, $items_per_page, $pelement);
-  
+
 The call to ``pager_default_initialize`` will return the current page.  The current page is a numeric number indicating which page the pager is currently showing. The first time the page is loaded this will always be the first page.  Each time the user navigates to other pages by clicking the "next" link or the numeric links then this ``view()`` function is called and the current page is set to the page being viewed. Next, we must theme the pager so that it follows the look-and-feel prescribed for the site. For this we use the Drupal ``theme()`` function.
 
 .. code-block:: php
@@ -114,13 +114,13 @@ The call to ``pager_default_initialize`` will return the current page.  The curr
     'parameters' => array(),
     'quantity' => $total_pages,
   ));
-  
+
 By default, all links in the pager cause the page to reload.  We do not want the page to reload, rather we only want to update the contents of the field.  The TripalFieldFormatter class provides a function named ``ajaxifyPager`` to convert a pager into an AJAX pager:
 
 .. code-block:: php
 
   $pager = $this->ajaxifyPager($pager, $entity);
-  
+
 Now that we have a pager, it has been setup for AJAX and we know the current page that the user is viewing we can now display only the items from the ``$items`` array that are appropriate for the page being viewed. A common way to provide multiple items on a page is within a table. When we set the ``$element`` array we need to be sure to provide both the content and the pager:
 
 .. code-block:: php
@@ -129,9 +129,9 @@ Now that we have a pager, it has been setup for AJAX and we know the current pag
       '#type' => 'markup',
       '#markup' => $content . $pager,
     );
-    
+
 The settingsForm() Function.
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Sometimes you may want to provide some control to the site developer for the formatter.  For example, the ``sbo__relationship_formatter`` allows the site developer to customize the title that appears above the table that houses relationships and the text the appears if there are no relationships.  By default the title is "Relationships" and the empty text indicates there are no relationships. Both are a bit too generic.  The ``settingsForm()`` function allows you to provide a Drupal form for the field that appears on the **Administer > Structure > Tripal Content Types** on any content type's **manage display** page:
 
 .. image:: custom_formatter.settings.1.png
@@ -140,7 +140,7 @@ The form shown in the screenshot above is provided by the ``settingsForm()`` fun
 
 .. code-block:: php
   :linenos:
-  
+
   public function settingsForm($view_mode, $form, &$form_state) {
 
     $display = $this->instance['display'][$view_mode];
@@ -156,13 +156,13 @@ The form shown in the screenshot above is provided by the ``settingsForm()`` fun
       '#title' => 'Empty text',
       '#default_value' => array_key_exists('empty', $settings) ? $settings['empty'] : 'There are no relationships',
     );
-  
+
     return $element;
   }
-  
+
 The form is typical of any form.  Note, however that the ``#default_value`` is set using the current settings values.
 
-A settings form is useful but it only works when Drupal knows what settings you want for your field.  You must provide the settings names (e.g. "title" and "empty" in this case) when you  attach your field to a given content type (i.e. bundle).  You tell Drupal to attach this field to a content type using the ``hook_bundle_instances_info`` function.  See 
+A settings form is useful but it only works when Drupal knows what settings you want for your field.  You must provide the settings names (e.g. "title" and "empty" in this case) when you  attach your field to a given content type (i.e. bundle).  You tell Drupal to attach this field to a content type using the ``hook_bundle_instances_info`` function.  See
 the :doc:`create_instance` to learn more about this function.  Briefly, the ``display`` section of the info array for the ``sbo__relationship`` field contains the following settings for the ``display``:
 
 .. code-block:: php
@@ -181,8 +181,10 @@ the :doc:`create_instance` to learn more about this function.  Briefly, the ``di
 .. warning::
 
     In order for the ``settingsForm()`` implemented to be available on the "Manage Display" page, you must also implement ``settingsSummary()`` as described below.
+
 The settingsSummary() Function.
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 The ``settingsSummary()`` function provides a summary of the current settings values for a field on the **manage display** page.  The following shows the same relationship field from the previous section, but with the settings form closed, and a summary of the current values shown:
 
 .. image:: custom_formatter.settings.2.png
@@ -191,7 +193,7 @@ An example of the ``settingsSummary()`` function that generates the summary in t
 
 .. code-block:: php
   :linenos:
-  
+
   public function settingsSummary($view_mode) {
     $display = $this->instance['display'][$view_mode];
     $settings = $display['settings'];
@@ -204,5 +206,3 @@ An example of the ``settingsSummary()`` function that generates the summary in t
 
     return $summary;
   }
-
-  

docs/dev_guide/custom_web_services.rst

@@ -1,18 +1,17 @@
 Creating Custom Web Services
 ==============================
 
-
 Introduction
 -------------
 
-New in Tripal v3 are `RESTful web services <https://en.wikipedia.org/wiki/Representational_state_transfer>`_.  These web-services are designed to allow end-users to access data programmatically using any programming language of their choice.  Web services in Tripal v3 are provided by the **tripal_ws** module.  By default the module provides the "content" service.  This service provides access via a RESTful interface to all of the content published on a Tripal site.  It is meant to respond immediately as the site admin makes changes to published data and is expected to always provide access to the same data displayed on the site (nothing more and nothing less). 
+New in Tripal v3 are `RESTful web services <https://en.wikipedia.org/wiki/Representational_state_transfer>`_.  These web-services are designed to allow end-users to access data programmatically using any programming language of their choice.  Web services in Tripal v3 are provided by the **tripal_ws** module.  By default the module provides the "content" service.  This service provides access via a RESTful interface to all of the content published on a Tripal site.  It is meant to respond immediately as the site admin makes changes to published data and is expected to always provide access to the same data displayed on the site (nothing more and nothing less).
 
 
-Tripal v3 has been redesigned from Tripal v2 to be fully organized around controlled vocabularies.  All data made available via Tripal is expected to be described by the terms of a controlled vocabulary or ontology.  For example, all content types in Tripal are assigned a controlled vocabulary term that describes what the content is.  Additionally, each field of data attached to a content type also is described using a controlled vocabulary term.  If a field provides more than just a single data value (i.e. it provides a list or nested structured array of data of key/value pairs) then each of the keys for those pairs must also be a controlled vocabulary term.  This requirement allows Tripal to fully describe the data it houses to any other Tripal site and any other script or service that can read the web services.  As long as the client application understands the vocabulary term it will understand the meaning of the data.  Using controlled vocabulary terms to describe all data allows a Tripal site to participate in the `Semantic Web <https://en.wikipedia.org/wiki/Semantic_Web>`_. 
+Tripal v3 has been redesigned from Tripal v2 to be fully organized around controlled vocabularies.  All data made available via Tripal is expected to be described by the terms of a controlled vocabulary or ontology.  For example, all content types in Tripal are assigned a controlled vocabulary term that describes what the content is.  Additionally, each field of data attached to a content type also is described using a controlled vocabulary term.  If a field provides more than just a single data value (i.e. it provides a list or nested structured array of data of key/value pairs) then each of the keys for those pairs must also be a controlled vocabulary term.  This requirement allows Tripal to fully describe the data it houses to any other Tripal site and any other script or service that can read the web services.  As long as the client application understands the vocabulary term it will understand the meaning of the data.  Using controlled vocabulary terms to describe all data allows a Tripal site to participate in the `Semantic Web <https://en.wikipedia.org/wiki/Semantic_Web>`_.
 
-Finally, the Tripal RESTful services are meant to be discoverable.  In some cases, when a web services is designed, the only way to understand the structure of it and the operations that it provides are for a programmer to read online documentation for the service before she can write the client application.  However, to better support automatic data discovery without human intervention by a client the Tripal web services have been designed to be discoverable.  To this end, Tripal uses the  `Hydra Core Vocabulary <https://www.hydra-cg.com/spec/latest/core/>`_ specification.  It fully describes all of the services, their operations, and the resulting value.  A client application that understands the Hydra language can therefore learn  to use the web service without human intervention.  However, in practice, its a good idea to continue to provide online documentation for humans.  And this User's Guide :doc:`provides those instructions </user_guide/web_services>` for the default Tripal content service.  
+Finally, the Tripal RESTful services are meant to be discoverable.  In some cases, when a web services is designed, the only way to understand the structure of it and the operations that it provides are for a programmer to read online documentation for the service before she can write the client application.  However, to better support automatic data discovery without human intervention by a client the Tripal web services have been designed to be discoverable.  To this end, Tripal uses the  `Hydra Core Vocabulary <https://www.hydra-cg.com/spec/latest/core/>`_ specification.  It fully describes all of the services, their operations, and the resulting value.  A client application that understands the Hydra language can therefore learn  to use the web service without human intervention.  However, in practice, its a good idea to continue to provide online documentation for humans.  And this User's Guide :doc:`provides those instructions </user_guide/building/web_services>` for the default Tripal content service.
 
-This documentation provides instructions to construct your own custom web services and making that service available through Tripal's existing web service infrastructure.  This will enable your web service to be discoverable and to provide a consistent experience to end-users.  Before proceeding with the following instructions, please review the **Structure of a Web Service Response** section on the User's Guide :doc:`Web Services page </user_guide/web_services>`.
+This documentation provides instructions to construct your own custom web services and making that service available through Tripal's existing web service infrastructure.  This will enable your web service to be discoverable and to provide a consistent experience to end-users.  Before proceeding with the following instructions, please review the **Structure of a Web Service Response** section on the User's Guide :doc:`Web Services page </user_guide/building/web_services>`.
 
 Getting Started
 ----------------
@@ -30,12 +29,12 @@ When you are ready to begin construction of your web service, you must first cre
 
 	[module_name]/includes/TripalWebService
 
-Where [module_name] is the name of your custom module.  It is inside of this directory that you will place the code for your new web services.  Tripal is designed to recognize this directory, discover the web services described inside of it and to automatically make them available!  You need only program the service and Tripal does the rest.  
+Where [module_name] is the name of your custom module.  It is inside of this directory that you will place the code for your new web services.  Tripal is designed to recognize this directory, discover the web services described inside of it and to automatically make them available!  You need only program the service and Tripal does the rest.
 
 
-**Note:** When selecting a version number it is best practice to start with 0.1.  The first number represents the  "major" number and the second number is the "minor" number.  When minor bug fixes or changes to the service occur the minor number should be incremented. When major changes occur that affect the structure of the web service or it's operations then the major number should be incremented.    The service can be named whatever you like.   This class file should be named according to this schema with a **.inc** extension. 
+**Note:** When selecting a version number it is best practice to start with 0.1.  The first number represents the  "major" number and the second number is the "minor" number.  When minor bug fixes or changes to the service occur the minor number should be incremented. When major changes occur that affect the structure of the web service or it's operations then the major number should be incremented.    The service can be named whatever you like.   This class file should be named according to this schema with a **.inc** extension.
 
-For this tutorial, suppose we wanted to create a web service that allowed someone to interact with the Tripal Job queue.  We could name our new class the **TripalJobService_v0.1** class and we would create this class in the file: 
+For this tutorial, suppose we wanted to create a web service that allowed someone to interact with the Tripal Job queue.  We could name our new class the **TripalJobService_v0.1** class and we would create this class in the file:
 
 .. code-block:: bash
 
@@ -46,7 +45,7 @@ Within this file we will implement our class with the following structure:
 
 .. code-block:: php
 
-	
+
 	class TripalJobService_v0_1 extends TripalWebService {
 
 	  /**
@@ -139,14 +138,14 @@ Notice currently all this function does is call the parent getDocumentation func
 	  ]
 	}
 
-In the above array, notice the @id is URL that represents a unique identifier for the class.   The @type will always be 'hydra:Class' because we are documenting a resource.  Then there is information about the class defined using the 'hydra:title' and 'hydra:description'.  The 'subclassOf' is always set to 'hydra:Resource'.  Next is the list of supported operations for this resource.  Remember, in our design we only want to support the GET operation for our Jobs service, so just like in the example above, the method we will support is GET.  The key/value pairs for the GET method are described using Hydra terms.  
+In the above array, notice the @id is URL that represents a unique identifier for the class.   The @type will always be 'hydra:Class' because we are documenting a resource.  Then there is information about the class defined using the 'hydra:title' and 'hydra:description'.  The 'subclassOf' is always set to 'hydra:Resource'.  Next is the list of supported operations for this resource.  Remember, in our design we only want to support the GET operation for our Jobs service, so just like in the example above, the method we will support is GET.  The key/value pairs for the GET method are described using Hydra terms.
 
 For our services we need to provide the information to Tripal so that it can generate these Hydra JSON arrays that document our service.  Tripal provides some easy API functions to help with this.  The first is the **addDoc** member function.  This function will take as input the class details, operations and properties necessary to generate the documentation for our class.  First, lets use this function to document our Jobs Collection resource.  Below is sample code that will do this for us.
 
 
 .. code-block:: php
 
-	
+
 
  	public function getDocumentation() {
         $term = tripal_get_term_details('local', 'computational_jobs');
@@ -167,7 +166,7 @@ For our services we need to provide the information to Tripal so that it can gen
         );
         $this->addDocClass($details, $operations, $properties);
         return parent::getDocumentation();
-    
+
 
 
 In the code above we add the documentation for our Job Collection class. There are three different arrays, one for the class details, one for the operations that the class supports and the third for properties. For now, the properties array is left empty. We'll come back to that later.  All classes must use a controlled vocabulary term.  Notice that the term used for this class is a term local to the database named 'computational_jobs'.   Normally when creating a class we would try to use a term from a published controlled vocabulary.  A large number of these vocabularies can be searched using `the EBI Ontology Lookup Service <https://www.ebi.ac.uk/ols/index>`_.  Unfortunately, an appropriate term could not be found in a published vocabulary, so we had to create a local term.  We can use Tripal's API functions to easily add new terms.  The following code should be placed in the install() function of your module to ensure the term is available:
@@ -219,7 +218,7 @@ in the code above need to determine if the resource is a job collection or a job
 
 .. code-block:: php
 
-	
+
 
     /**
      * Generates the job collection resource.
@@ -253,7 +252,7 @@ in the code above need to determine if the resource is a job collection or a job
         }
     }
 
-The first few lines of code above are as follows:  
+The first few lines of code above are as follows:
 
 .. code-block:: php
 
@@ -294,7 +293,7 @@ Lastly, we need to add our "members" of the collection.  These are the jobs from
 
 .. code-block:: php
 
-	
+
 
         foreach ($jobs as $job) {
             $member = new TripalWebServiceResource($service_path);
@@ -305,7 +304,7 @@ Lastly, we need to add our "members" of the collection.  These are the jobs from
             $this->resource->addMember($member);
         }
 
-Notice in the code above that each job is an instance of the class called TripalWebServiceResource.  We use this class because each element of the collection is a reference to a resource and we reference the ID and the type.   In the code above we create the new member resource, we set it's type to be the vocabulary term 'local:computational_job' and eventually, use the addMember function of our TripalWebServiceCollection to add it to the collection. 
+Notice in the code above that each job is an instance of the class called TripalWebServiceResource.  We use this class because each element of the collection is a reference to a resource and we reference the ID and the type.   In the code above we create the new member resource, we set it's type to be the vocabulary term 'local:computational_job' and eventually, use the addMember function of our TripalWebServiceCollection to add it to the collection.
 
 Also in the code above is a new function named addProperty.   We want to add some additional information about the job to help the end-user understand what each job is and how to get to it.  Here we add two properties, one that is the job name and another that is the page URL for the job on our Tripal site.  With these properties the client can quickly see the title and can go see the job on the Tripal site by following the given URL.  Note, a resource always has two identifying pieces of information: the ID and the Type.  So, everything else is added as a property of the resource.   Also notice that the first argument when using the addProperty function is a controlled vocabulary term.  Here we've used the terms **schema:name** and **schema:ItemPage**.  These terms are from the Schema.org vocabulary and the define what these properties are: a name and an item's page.
 
@@ -328,7 +327,7 @@ Simplifying the Property Keys
 
 Implementing a Resource
 ------------------------
- 
+
 Documenting Properties
 ------------------------
 
@@ -342,6 +341,6 @@ Controlling Access to Resources
 ---------------------------------
 
 
- 
 
-Debugging and Troubleshooting
+
+Debugging and Troubleshooting

docs/extensions/module_rating.rst

@@ -1,11 +1,10 @@
-
 Tripal Module Rating System
-=============================
+===========================
 
 This module rating system is meant to aid Tripal Site Administrators in choosing extension modules for their site. It is also meant to guide developers in module best practices and celebrate modules which achieve these goals.
 
 Bronze
--------
+------
 
 .. image:: Tripal-Bronze.png
 
@@ -17,7 +16,7 @@ Bronze
 - Has a license (distributed with module).
 
 Silver
--------
+------
 
 .. image:: Tripal-Silver.png
 
@@ -31,7 +30,7 @@ Silver
     - e.g. first release for Drupal 7 should be: ``7.x-1.0``.
 
 Gold
------
+----
 
 .. image:: Tripal-Gold.png
 

docs/user_guide.rst

@@ -1,9 +1,9 @@
 Tripal v3 User's Guide
 ======================
 
-.. note:: 
+.. note::
 
-  Looking for the `Tripal v2 User's Guide <http://tripal.info/tutorials/v2.x>`_? 
+  Looking for the `Tripal v2 User's Guide <http://tripal.info/tutorials/v2.x>`_?
 
 
 .. toctree::
@@ -12,16 +12,9 @@ Tripal v3 User's Guide
    :glob:
 
    user_guide/what_is_tripal
+   user_guide/prereqs
    user_guide/install_tripal
-   user_guide/drupal_overview
-   user_guide/learn_chado
-   user_guide/content_types
-   user_guide/example_genomics
-   user_guide/file_management
-   user_guide/mviews
-   user_guide/custom_tables
-   user_guide/searching
-   user_guide/job_management
-   user_guide/web_services
-   user_guide/bulk_loader
-   user_guide/customize_site
+   user_guide/building
+   user_guide/customize
+   user_guide/admin
+   user_guide/tutorials

docs/user_guide/admin.rst

@@ -0,0 +1,11 @@
+Site Administration
+===================
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Table of Contents
+   :glob:
+
+   ./admin/jobs
+   ./admin/file_management
+   ./admin/upgrade_tripal

docs/user_guide/file_management.rst → docs/user_guide/admin/file_management.rst

@@ -1,36 +1,36 @@
-File Management
-===============
+Files
+=====
 
 User Quotas
 -----------
 Data importers that use the Tripal API and Tripal supported widgets automatically associate uploaded files with users. If you are allowing end-users to upload files you may want to consider adding quotas to prevent the server storage from filling.  To ensure that users do not exceed the limits of the server a quota system is available.  Navigate to **Administer > Tripal > User File Management** and click the **User Quotas** tab to reveal the following page:
 
-.. image:: ./file_management.user_quotas.1.png
+.. image:: ./file_management/file_management.user_quotas.1.png
 
 First, the total amount of space consumed by all uploaded files is shown at the top of the page.  Initially this will indicate 0 B (for zero bytes); as users upload files this value will change.  You may return to this page in the future to check how much space is currently used by user uploads. Here you can also specify the default system-wide quota that all users receive.  By default this is set to 64 Megabytes and an expiration of 60 days.  Once a file has existed on the site for 60 days the file is marked for deletion and will be removed when the Drupal cron is executed.  The default of 64MB per user is most likely too small for your site.  Adjust this setting and the days to expire as appropriate for your site's expected number of users and storage limitations and click the **Save** button to preserve any changes you have made.
 
 In addition to the default settings for all users, you may want to allow specific users to have a larger (or perhaps smaller) quota.  You can set user-specific quotas by clicking the **Add Custom User Quota** link near the bottom of the page.   The following page appears:
 
-.. image:: ./file_management.user_quotas.2.png
+.. image:: ./file_management/file_management.user_quotas.2.png
 
 
 Here you must specify the Drupal user name of the user who should be granted a custom quota.  This field will auto populate suggestions as you type to help you find the correct username.  Enter the desired quota size and expiration days and click the **Submit** button. you will then see the user-specific quota listed in the table at the bottom of the page:
 
-.. image:: ./file_management.user_quotas.3.png
+.. image:: ./file_management/file_management.user_quotas.3.png
 
 User's Files
 ------------
 User's with permission to upload files are able to use the Tripal file uploader to add files to the server.  The core Tripal Data Importers use the Tripal file uploader and extension modules may use it as well.  You can enable this functionality for users by Navigating to **Admin** > **People** and click the **Permissions** Tab. next scrol to the **Tripal** section and set the **Upload Files** permissions as desired for your site.  The following screenshot shows the permission on a default Drupal site.
 
 
-.. image:: ./file_upload_permission.png.png
+.. image:: ./file_management/file_upload_permission.png
 
-User's who have the ability to upload files can manage files on their own Account pages.  
+User's who have the ability to upload files can manage files on their own Account pages.
 
 As described in the previous section, the site administrator can set a system-wide or user-specific default expiration number of days for a file. This means files will be removed automatically from the server once their expiration data is set.
 
 .. note::
 
   Automatic removal of files can only occur if the Drupal cron is setup to run automatically.
-  
-Each  
+
+Each

docs/user_guide/admin/jobs.rst

@@ -0,0 +1,10 @@
+Jobs
+====
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Table of Contents
+   :glob:
+
+   ./jobs/automating_job_execution
+   ./jobs/tripal_daemon

docs/user_guide/install_tripal/automating_job_execution.rst → docs/user_guide/admin/jobs/automating_job_execution.rst

@@ -3,7 +3,7 @@ Automating Job Execution
 
 .. note::
 
-  Remember you must set the ``$DRUPAL_HOME`` environment variable if you want to cut-and-paste the commands below. See :doc:`./drupal_home`
+  Remember you must set the ``$DRUPAL_HOME`` environment variable if you want to cut-and-paste the commands below. See :doc:`../../prereqs/drupal_home`
 
 The Drupal cron is used to automatically execute necessary Drupal housekeeping tasks on a regular interval.  You should *always* setup the Drupal cron to ensure your site checks for updates and security issues.  To do this, we want to integrate Drupal cron with the UNIX cron facility.  The UNIX cron will automatically execute commands on set regular intervals.  First, we must get the appropriate URL for the cron by navigating to **Configuration → Cron**. On this page you will see a link that we will use for cron:
 
@@ -69,5 +69,3 @@ Tripal version 3 has incorporated the Tripal Daemon module.  This module was pre
 .. code-block:: bash
 
   drush pm-enable tripal_daemon
-
-Further documentation for setup of the Tripal Daemon will appear here in the future.  For now, please see the :doc:`Job Management  </user_guide/job_management>` page for usage instructions.

docs/user_guide/job_management.rst → docs/user_guide/admin/jobs/tripal_daemon.rst

@@ -1,9 +1,9 @@
-Job Management (Tripal Daemon)
-==============================
+Tripal Daemon
+=============
 
 .. note::
 
-  Remember you must set the $DRUPAL_HOME environment variable to cut-and-paste the commands below. See see :doc:`./install_tripal/drupal_home`
+  Remember you must set the $DRUPAL_HOME environment variable to cut-and-paste the commands below. See see :doc:`../../prereqs/drupal_home`
 
 
 The Tripal Daemon module is meant to provide a simple means of creating a robust command-line-driven, fully bootstrapped PHP Daemon. It uses the PHP-Daemon (https://github.com/shaneharter/PHP-Daemon) Library to create the Daemon (via the Libraries API) in order to not re-invent the wheel. It allows you to execute Jobs submitted to Tripal without using cron.  It provides a faster user experience for running jobs.  Prior to Tripal v3, the Tripal Daemon module was an extension module. It was integrated into the core Tripal package.

docs/user_guide/admin/upgrade_tripal.rst

@@ -0,0 +1,11 @@
+Upgrade Tripal
+===============
+
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Table of Contents
+   :glob:
+
+   ./upgrade_tripal/updating
+   ./upgrade_tripal/upgrade_from_tripal2

docs/user_guide/admin/upgrade_tripal/updating.rst

@@ -0,0 +1,2 @@
+Updating Tripal and Drupal
+==========================

docs/user_guide/install_tripal/upgrade_from_tripal2.rst → docs/user_guide/admin/upgrade_tripal/upgrade_from_tripal2.rst

@@ -1,5 +1,5 @@
 Upgrade from Tripal v2 to v3
-================================
+============================
 
 .. note::
 
@@ -25,12 +25,12 @@ Step 1: Upgrade Tripal
   .. warning::
 
     If you have made customizations to Chado you may encounter problems during the upgrade.  It is not recommended to ever change any of the existing tables of Chado. However, if you have and if you do encounter such issues, please use the Tripal Issue queue to request help: https://github.com/tripal/tripal/issues
-    
+
     If you have custom Drupal fields attached to Tripal nodes then the content in those fields will not automatically be migrated to the new Tripal v3 entities. Bradford Condon has provided some instructions to help migrate these fields after the site has been upgrade. You can find those instructions `here <https://gist.github.com/bradfordcondon/0dddfd015ff6ef1f545364c2ceff1f0b>`_.
 
 2. Put the site in maintenance mode. Before completing any upgrade you should put your site into "maintenance mode". This ensures that users are isolated from any temporary error messages generated through the process. To put the site in maintenance mode, navigate to **Administration > Configuration > Maintenance Mode** . Then click the **Put site into maintenance mode** checkbox and click **Save Configuration**. Additionally, there is a text area on this page that allows you to customize the message displayed to your users while your site is in maintenance mode.
 
-  .. image:: upgrade_from_tripal2.step1-2.png
+  .. image:: ./upgrade_from_tripal2/upgrade_from_tripal2.step1-2.png
 
   You can also put your site into "Maintenance mode" using drush be executing the following command:
 
@@ -100,7 +100,7 @@ Step 1: Upgrade Tripal
 
 10. Return to your Tripal site, and click the link that appears for preparing Chado and launch the job.
 
-  .. image:: upgrade_from_tripal2.step1-10.png
+  .. image:: ./upgrade_from_tripal2/upgrade_from_tripal2.step1-10.png
 
 
   .. note::
@@ -129,11 +129,11 @@ The process allows you to create Tripal 3 content types exposing the same data a
 
 1. Navigate to **Administration > Tripal > Data Storage > Chado** and click on Step 2.
 
-  .. image:: upgrade_from_tripal2.step2-1.png
+  .. image:: ./upgrade_from_tripal2/upgrade_from_tripal2.step2-1.png
 
 2. Select an individual content type to migrate from the Tripal v2 Content Type drop-down.
 
-  .. image:: upgrade_from_tripal2.step2-2.png
+  .. image:: ./upgrade_from_tripal2/upgrade_from_tripal2.step2-2.png
 
 3. Click the 'Get Tripal v3 Types' button to retrieve a list of Tripal v3 content types to which this Tripal v2 type can be converted. This may take a while depending on the size of your database.
 
@@ -159,7 +159,7 @@ That said, if you decide to stick with your current customized templates, the fo
 
 1. Navigate to **Administration > Tripal > Data Storage > Migrate** and click on Step 3
 
-  .. image:: upgrade_from_tripal2.step3-1.png
+  .. image:: ./upgrade_from_tripal2/upgrade_from_tripal2.step3-1.png
 
 2. Click the checkbox for the Tripal v2 content types you want to keep your old templates for. Unchecked content types will use the new Tripal 3 interface.
 
@@ -172,7 +172,7 @@ This final step allows you to fully switch to Tripal v3 pages. You can move URLs
 
 1. Navigate to **Administration > Tripal > Data Storage > Migrate** and click on Step 4
 
-  .. image:: upgrade_from_tripal2.step4-1.png
+  .. image:: ./upgrade_from_tripal2/upgrade_from_tripal2.step4-1.png
 
 2. Once you have confirmed that you are happy with the Tripal v3 pages for a given content type, check the desired check boxes for that content type.
 
@@ -210,8 +210,8 @@ You can avoid this problem by clearing out the Drupal search tables byu executin
 
   To do this, go to **Configuration > Text formats** in your administrative menu and click on the 'Add text format' link:
 
-  .. image:: upgrade_from_tripal2.troub-1-1.png
+  .. image:: ./upgrade_from_tripal2/upgrade_from_tripal2.troub-1-1.png
 
   Make sure its machine-readable_name is 'full_html' and save the configuration.
 
-  .. image:: upgrade_from_tripal2.troub-1-2.png
+  .. image:: ./upgrade_from_tripal2/upgrade_from_tripal2.troub-1-2.png

docs/user_guide/building.rst

@@ -0,0 +1,14 @@
+Build Your Site
+===============
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Table of Contents
+   :glob:
+
+   ./building/content_types
+   ./building/mviews
+   ./building/custom_tables
+   ./building/searching
+   ./building/web_services
+   ./building/bulk_loader

docs/user_guide/bulk_loader.rst → docs/user_guide/building/bulk_loader.rst

@@ -3,9 +3,9 @@ Bulk Loader
 
 .. note::
 
-  Remember you must set the ``$DRUPAL_HOME`` environment variable if you want to cut-and-paste the commands below. See :doc:`./install_tripal/drupal_home`
+  Remember you must set the ``$DRUPAL_HOME`` environment variable if you want to cut-and-paste the commands below. See :doc:`../prereqs/drupal_home`
 
-The bulk loader is a tool that Tripal provides for loading of data contained in tab delimited files. Tripal supports loading of files in standard formats (e.g. ``FASTA``, ``GFF``, ``OBO``), but Chado can support a variety of different biological data types and there are often no community standard file formats for loading these data. For example, there is no file format for importing genotype and phenotype data. Those data can be stored in the feature, stock and natural diversity tables of Chado. The Bulk Loader was introduced in Tripal v1.1 and provides a web interface for building custom data loader. In short, the site developer creates the bulk loader "template". This template can then be used and re-used for any tab delimited file that follows the format described by the template. Additionally, bulk loading templates can be exported allowing Tripal sites to share loaders with one another.  
+The bulk loader is a tool that Tripal provides for loading of data contained in tab delimited files. Tripal supports loading of files in standard formats (e.g. ``FASTA``, ``GFF``, ``OBO``), but Chado can support a variety of different biological data types and there are often no community standard file formats for loading these data. For example, there is no file format for importing genotype and phenotype data. Those data can be stored in the feature, stock and natural diversity tables of Chado. The Bulk Loader was introduced in Tripal v1.1 and provides a web interface for building custom data loader. In short, the site developer creates the bulk loader "template". This template can then be used and re-used for any tab delimited file that follows the format described by the template. Additionally, bulk loading templates can be exported allowing Tripal sites to share loaders with one another.
 
 The following commands can be executed to install the Tripal Bulk Loader using Drush:
 
@@ -13,7 +13,7 @@ The following commands can be executed to install the Tripal Bulk Loader using D
 
   cd /var/www/
   drush pm-enable tripal_bulk_loader
-  
+
 
 Plan How to Store Data
 ----------------------
@@ -102,11 +102,11 @@ Creating a New Bulk Loader Template
 
 Now that we know where all of the data in the input file will go and we have the necessary dependencies in the database (i.e. the NCBI Taxonomy database), we can create a new bulk loader template. Navigate to ``Tripal → Data Loaders → Chado Bulk Loader``, click the **Templates** tab in the top right corner, and finally click the link **Add Template**. The following page appears:
 
-.. image:: ./bulk_loader.1.png
+.. image:: ./bulk_loader/bulk_loader.1.png
 
 We need to first provide a name for our template. Try to name templates in a way that are meaningful for others. Currently only site administrators can load files using the bulk loader. But, future versions of Tripal will provide functionality to allow other privileged users the ability to use the bulk loader templates. Thus, it is important to name the templates so that others can easily identify the purpose of the template. For this example, enter the name **NCBI Taxonomy Importer (taxid, genus, species)**. The following page appears:
 
-.. image:: ./bulk_loader.2.png
+.. image:: ./bulk_loader/bulk_loader.2.png
 
 Notice that the page is divided into two sections: **Current Records** and **Current Fields**. Before we continue with the template we need a bit of explanation as to the terminology used by the bulk loader. A **record** refers to a Chado table and an action on that table. For example, to insert the data from the input file we will need to select the NCBI Taxonomy database from the **db** table and insert entries into the **dbxref**, **organism** and **dbxref_organism** tables. Therefore, we will have four records:
 
@@ -119,7 +119,7 @@ Each record contains a set of fields on which the action is performed. Thus, whe
 
 To create the first record for inserting an organism, click the button **New Record/Field**. The following page appears:
 
-.. image:: ./bulk_loader.3.png
+.. image:: ./bulk_loader/bulk_loader.3.png
 
 By default, when adding a new record, the bulk loader also provides the form elements for adding the first field of the record as well. We are adding a new record, so we can leave the **Record** drop-down as **New Record**. Next, give this record a unique record name. Because we are inserting into the organism table, enter the name **Organism** into the **Unique Record Name** box.
 
@@ -127,7 +127,7 @@ We also have the opportunity with this form to add our first field to the record
 
 In the next section, titled **Data File Column**, we need to indicate the column in the tab-delimited file where the genus is found. For the example file this is column 2 (columns are ordered beginning with number 1). Therefore, enter the number **2** in the **Column** box. There are additional options to expose the field to the user, but for now we can ignore those options. Click the **Save Changes** button at the bottom. We now see that the organism record and the first field have been added to our bulk loader template.
 
-.. image:: ./bulk_loader.4.png
+.. image:: ./bulk_loader/bulk_loader.4.png
 
 We also see that the **Mode** (or action) for this record has been set to insert by default. Before continuing we should edit the settings for the record so that it is more fault tolerant. Click the **Edit** link to the left of the new organism record. On the resulting page we see the record details we already provided, but now there is a section titled **Action to take when Loading Record**. By default, the **INSERT** option is selected. This is correct. We want to perform an insert. However, notice in the **Additional Insert Options** section, the **SELECT if duplicate (no insert).** Check this box. This is a good option to add because it prevents the bulk loader from failing if the record already exists in the table.
 
@@ -143,7 +143,7 @@ Next, we need to add the **species** field to the record. Click the **Add Field*
 
 We now have two fields for our organism record:
 
-.. image:: ./bulk_loader.5.png
+.. image:: ./bulk_loader/bulk_loader.5.png
 
 At this point our organism record is complete, however there are still a few fields in the organism table of Chado that are not present in our record. These include the **organism_id, abbreviation, common_name** and **comment** fields. We do not have values in our input file for any of these fields. Fortunately, the **organism_id** field is a primary key field and is auto generated when a record is submitted. We do not need to provide a value for that field. The other fields are not part of the unique constraint of the table. Therefore, those fields are optional and we do not need to specify them. Ideally, if we did have values for those non-required fields we would add them as well.
 
@@ -184,7 +184,7 @@ Now that we have a record that selects the **db_id** we can now create the **dbx
 
 Click the Save Changes button. The Edit Template page appears.
 
-.. image:: ./bulk_loader.6.png
+.. image:: ./bulk_loader/bulk_loader.6.png
 
 Again, we need to edit the record to make the loader more fault tolerant. Click the Edit link to the left of the Taxonomy ID record. Select the following:
 
@@ -233,7 +233,7 @@ Create the second field:
 
 We are now done! We have created a bulk loader template that reads in a file with three columns containing an NCBI taxonomy ID, a genus and species. The loader places the genus and species in the **organism** table, adds the NCBI Taxonomy ID to the **dbxref** table,  links it to the NCBI Taxonomy entry in the db table, and then adds an entry to the **organism_dbxref** table that links the organism to the NCBI taxonomy Id. The following screen shots show how the template should appear:
 
-.. image:: ./bulk_loader.7.png
+.. image:: ./bulk_loader/bulk_loader.7.png
 
 To save the template, click the **Save Template** link at the bottom of the page.
 
@@ -242,7 +242,7 @@ Creating a Bulk Loader Job (importing a file)
 
 Now that we have created a bulk loader template we can use it to import a file. We will import the **Fragaria**.txt file downloaded previously. To import a file using a bulk loader template, click the **Add Content** link in the administrative menu and click the **Bulk Loading Job**. A bulk loading job is required each time we want to load a file. Below is a screen shot of the page used for creating a bulk loading job.
 
-.. image:: ./bulk_loader.8.png
+.. image:: ./bulk_loader/bulk_loader.8.png
 
 Provide the following values:
 
@@ -258,11 +258,11 @@ Provide the following values:
 
 Click **Save**. The page then appears as follows:
 
-.. image:: ./bulk_loader.9.png
+.. image:: ./bulk_loader/bulk_loader.9.png
 
 You can see details about constants that are used by the template and the where the fields from the input file will be stored by clicking the **Data Fields** tab in the table of contents on the left sidebar.
 
-.. image:: ./bulk_loader.10.png
+.. image:: ./bulk_loader/bulk_loader.10.png
 
 Now that we have created a job, we can submit it for execution by clicking the **Submit Job** button. This adds a job to the Tripal Jobs systems and we can launc the job as we have previously in this tutorial:
 
@@ -300,7 +300,7 @@ After execution of the job you should see similar output to the terminal window:
 
 Our *Fragaira* species should now be loaded, and we return to the Tripal site to see them. Click on the **Organisms** link in the **Search Data** menu.  In the search form that appears, type "Fragaria" in the **Genus** text box and click the **Filter** button. We should see the list of newly added *Fragaria* species.
 
-.. image:: ./bulk_loader.11.png
+.. image:: ./bulk_loader/bulk_loader.11.png
 
 Before the organisms will have Tripal pages, the Chado records need to be **Published**.  You can publish them by navigating to **Tripal Content -> Publish Tripal Content**.  Select the **organism** table from the dropdown and run the job.
 
@@ -310,7 +310,7 @@ Before the organisms will have Tripal pages, the Chado records need to be **Publ
 
 Once complete, return to the search form, find a *Fragaria* species that has been published and view its page. You should see a Cross References link in the left table of contents. If you click that link you should see the NCBI Taxonomy ID with a link to the page:
 
-.. image:: ./bulk_loader.12.png
+.. image:: ./bulk_loader/bulk_loader.12.png
 
 
 Sharing Your Templates with Others
@@ -318,7 +318,7 @@ Sharing Your Templates with Others
 
 Now that our template for loading organisms with NCBI Taxonomy IDs is completed we can share our template loader with anyone else that has a Tripal-based site.  To do this we simply export the template in text format, place it in a text file or directly in an email and send to a collaborator for import into their site.  To do this, navigate to **Tripal → Chado Data Loaders → Buik Loader** and click the **Tempalate** tab at the top.  Here we find a table of all the templates we have created.  We should see our template named **NCBI Taxonomy Importer** (taxid, genus, species).  In the far right colum is a link to export that template.  Licking that link will redirect you to a page where the template is provided in a serialized PHP array.
 
-.. image:: ./bulk_loader.13.png
+.. image:: ./bulk_loader/bulk_loader.13.png
 
 Cut-and-paste all of the text in the **Export** field and send it to a collaborator.
 

docs/user_guide/custom_tables.rst → docs/user_guide/building/custom_tables.rst

@@ -3,7 +3,7 @@ Custom Tables
 
 Chado has a large number of tables that can be used for storing a variety of biological and ancillary data.  All effort should be made to use the existing tables for storing data. However, there are times when the current set of tables is not adequate for storing some data. Tripal allows you to create custom tables in Chado using the Tripal web interface.  The interface for managing custom tables can be found on the **Tripal → Data Storage → Chado -> Custom Tables** page.
 
-.. image:: ./custom_tables.1.png
+.. image:: ./custom_tables/custom_tables.1.png
 
 This page contains a link to add a new custom table (at the top), a search interface to find tables by name, and a listing of existing tables.  You can use this interface to add, edit and remove custom tables.  By default, several custom tables have been created automatically by Tripal modules.  These tables hold data managed by the modules.  Also, tables used as materialized views appear in this list as well. They are identified with the `Is Mview` column.
 
@@ -19,9 +19,9 @@ If you need to create a new table you should follow the following procedure.
 4.  Write your table in the `Drupal Schema API <https://www.drupal.org/docs/7/api/schema-api/introduction-to-schema-api>`_ array format.
 5.  Navigate to the  **Tripal → Data Storage → Chado -> Custom Tables** page on your site. Click the `Add Custom Table` link at the top and cut-and-paste the schema array and click the `Add` button.
 
-.. image:: ./custom_tables.2.png
+.. image:: ./custom_tables/custom_tables.2.png
 
-One your table is created, you will want to add data to it.  You can do so without any programming by using the :doc:`./bulk_loader`.  The bulk loader expects that your data is housed in a tab-delimited text file and allows you to map columns from your file to columns in your new custom table. Each row of your file becomes a new record in your table.  
+One your table is created, you will want to add data to it.  You can do so without any programming by using the :doc:`./bulk_loader`.  The bulk loader expects that your data is housed in a tab-delimited text file and allows you to map columns from your file to columns in your new custom table. Each row of your file becomes a new record in your table.
 
-Alternatively, if you can write custom loaders using the Tripal API as described in the 
-:doc:`../dev_guide`
+Alternatively, if you can write custom loaders using the Tripal API as described in the
+:doc:`../../dev_guide`

docs/user_guide/mviews.rst → docs/user_guide/building/mviews.rst

@@ -3,13 +3,13 @@ Materialized Views
 
 .. note::
 
-  Remember you must set the ``$DRUPAL_HOME`` environment variable if you want to cut-and-paste the commands below. See :doc:`./install_tripal/drupal_home`
+  Remember you must set the ``$DRUPAL_HOME`` environment variable if you want to cut-and-paste the commands below. See :doc:`../prereqs/drupal_home`
 
 Chado is efficient as a data warehouse but queries can become slow depending on the type of query. To help simplify and speed up these queries, materialized views can be employed. For a materialized view, a new database table is created and then populated with the results of a pre-defined SQL query. This allows you to execute a much simpler and faster query on the materialized view when producing user pages. A side effect, however is redundant data, with the materialized view becoming stale if not updated regularly.
 
 Tripal provides a mechanism for populating and updating these materialized views. These can be found on the **Tripal → Data Storage → Chado -> Materialized Views** page.
 
-.. image:: ./mviews.1.png
+.. image:: ./mviews/mviews.1.png
 
 Here we see several materialized views. These were installed automatically by the Tripal Chado module. To update these views, click the **Populate** link for each one.
 

docs/user_guide/web_services.rst → docs/user_guide/building/web_services.rst

@@ -3,7 +3,7 @@ Web Services
 
 .. note::
 
-  Remember you must set the ``$DRUPAL_HOME`` environment variable if you want to cut-and-paste the commands below. See :doc:`./install_tripal/drupal_home`
+  Remember you must set the ``$DRUPAL_HOME`` environment variable if you want to cut-and-paste the commands below. See :doc:`../prereqs/drupal_home`
 
 Overview
 --------
@@ -43,7 +43,7 @@ Exploring Web Services
 
 Once enabled, web services are immediately available on your site at the URL  ``http://[your.site.name]/web-services/`` (replace [your.site.name] with the address and path of your Tripal site). Web services are meant to be accessed programmatically, but they can be easily explored using a web browser such as with the `Firefox browser <https://www.mozilla.org/en-US/firefox/>`_ and `JSONView <https://jsonview.com/>`_ extension enabled.  For example, the following screen shot shows an example Tripal site with the data loaded following the Setup of an Example Genomics Site instructions of this User's Guide.
 
-.. image:: web_services.1.png
+.. image:: ./web_services/web_services.1.png
 
 This initial resource "home page" of the web services returns results in `JSON format <http://www.json.org/>`_.  When using the JSONView extension within Firefox you can explore web services by clicking the links that are present in the browser.
 
@@ -103,7 +103,7 @@ The Content Service
 
 The content service provided by Tripal shares all publicly available content.  The content that appears on a page is the same content that appears in web services.  A major change in the design of Tripal from v2 to v3 is that all data is organized via controlled vocabularies.  The following diagram demonstrates how this is implemented.  For example the mRNA term comes from the `Sequence Ontology <http://www.sequenceontology.org/>`_.  It's unique term accession is SO:0000234.  Every content type in Tripal consists solely of a type (e.g. mRNA or SO:0000234), it's associated label (e.g.  mRNA) and a numeric id unique to each Tripal site.  The ID is what uniquely identifies every content in Tripal.  Each unique content with these three attributes is referred to as an **Entity**.  All other data associated with a given entity are called **Fields**.  Example fields for an mRNA content type may be the name of the mRNA, a unique name, the coding sequence, the coordinates on the reference genome, etc.  In the diagram below, these fields are the rectangular boxes that jut out of the mRNA entity.   These fields can be "attached" to an entity by Tripal and data can come from any storage backend.  The data that appears on a page and the data in the content service is taken from the same entity and therefore end-users and clients have access to the same data.
 
-.. image:: ./web_services.2.png
+.. image:: ./web_services/web_services.2.png
 
 
 Content Type Listing
@@ -111,7 +111,7 @@ Content Type Listing
 
 When the content service is accessed, the response is always a listing of all available content types on the site.   Site administrators can create new content types by following the Create Content Types section of this tutorial.  By default, Tripal is installed with several commonly used content types, but new ones can be created as needed for the site.  Whenever a new content type is created it immediately is available via the content service, and these content types can be found at the path:  ``/web-services/content/v0.1``.  Below is an example screenshot of the resulting JSON from an example site:
 
-.. image:: ./web_services.3.png
+.. image:: ./web_services/web_services.3.png
 
 Note that the **@type** for this listing is a Collection and the label is **Content Types**.  Each content type has a unique **@id**, a **@type** indicating the term that describes it and a brief description.  The **@id** serves as a URL to obtain further details about that content type.   Also, notice in the above screenshot that the **@context** section is minimized, but as usual, each of the terms used in the key/value pairs are fully qualified in that section.   This JSON-LD response also indicates the total number of content types available.
 
@@ -120,7 +120,7 @@ Content Type Members (Entities)
 
 The members or entities that belong to a content type are found at the path:  ``/web-services/content/v0.1/{name}`` where {name} is the name of the content type.  The {name} field must be identical to the label field from the content type listing shown previously.   For example, the mRNA content type  path would be   ``/web-services/content/v0.1/mRNA``.  This resource provides a listing of all members for that content type.   The following shows the response for an mRNA listing:
 
-.. image:: ./web_services.4.png
+.. image:: ./web_services/web_services.4.png
 
 Note that the **@type** is also a Collection byt the label is 'mRNA collection'.  To maintain a small response, the results of content member listings is usually paged such that only a subset of members is shown.  In this example, there are 8032 mRNA entities available, but only 25 are shown.  Notice the view term.  It contains several sub elements named first, last and next. These provide navigation links that can be used by a client application to iterate through all entities.  The structure of these links is as follows:
 
@@ -135,11 +135,11 @@ Content (Entity) Resource
 
 Each entity is accessible via the path: ``/web-services/content/v0.1/{name}/{id}``.   Here {name} continues to refer to the name of the content type (e.g. mRNA) and {id} refers to the unique numeric ID for the entity.  In this example an mRNA entity would result in a JSON-LD response like the following:
 
-.. image:: ./web_services.5.png
+.. image:: ./web_services/web_services.5.png
 
 In the JSON shown above, note that all of the key/value pairs used are referenced in the **@context** section  Also, notice that some key/value pairs contain data while others contain URLs.  Tripal is optimized to not load every attribute.  For example, sequence data for some content type may be large.  By providing a URL for the data, it keeps the response small but allows clients to access that information via the provided URL.   For example, if the URL for the **sequence_coordinate** attribute were followed the following response could be seen:
 
-.. image:: ./web_services.6.png
+.. image:: ./web_services/web_services.6.png
 
 Here the client can obtain the necessary information about the coordinates on the genome for this particular mRNA entity.