Merge branch 'rtd' of github.com:tripal/tripal into rtd

docs/dev_guide.rst

@@ -3,8 +3,47 @@ Developer's Guide
 
 
 .. toctree::
-   :maxdepth: 1
+   :maxdepth: 2
    :caption: Table of Contents
    :glob:
 
-   dev_guide/*
+   dev_guide/data_structures
+   dev_guide/best_practices
+   dev_guide/custom_modules
+   dev_guide/custom_field
+   dev_guide/custom_data_loader
+   dev_guide/custom_web_services
+
+
+
+
+Introduction to the Tripal API
+==============================
+
+Tripal provides an Application Programming Interfaces (API) that allows developers to interact with and customize Tripal. Using the API, developers can customize the way data is presented or create custom modules that provide new or different functionality. These custom modules can in turn be shared with the Tripal community. The Tripal API is documented online at http://api.tripal.info/api/tripal/3.x (COMING SOON). This document provides examples and best practices for using the Tripal API. 
+
+Requirements
+==============================
+
+In order to use the Tripal API the developer should have the following skills: 
+
+Common skills:
+
+* Knowledge of PHP.
+* Knowledge of relational databases and SQL.
+
+To create custom modules:
+
+* Familiarity with Drupal API
+* Familiarity with Drupal module development
+
+To use Chado for data storage:
+
+* Knowledge of Chado and relationships between tables (at least tables where data of interest is stored).
+* An idea how data is stored in Chado (or a willingness to ask questions)
+
+Things that will make your Tripal development experience postive, fun and rewarding:
+
+* A desire to write code that can be re-used and shared by other groups
+* A desire to share your work with others to support other the Tripal community members.
+* A willingness to ask for help on the Tripal Github issue queue if you get stuck, find bugs or desire new features.

docs/dev_guide/best_practices.rst

@@ -0,0 +1,64 @@
+Module Development Best Practice
+================================
+
+
+If you create custom Tripal Modules, here are some best practices and suggestions.
+
+The Drupal Devel Module
+=======================
+
+
+Before staring your development work, it is suggested that you download and install the `Drupal devel module <https://drupal.org/project/devel>`_. This module helps greatly with debugging your custom theme or module. A very useful function of this module is the dpm function. You can use the dpm function to print to the web page an interactive view of the contents of any variable. This can be extremely helpful when accessing Chado data in objects and arrays returned by Tripal.
+
+Add your module to Tripal.info
+==============================
+
+Do your best to list and upate your modules at http://tripal.info/extensions.
+
+
+Coding Best Practices
+=======================
+
+Host your code on GitHub
+-------------------------
+
+We recommend making your code open source and hosting it on GitHub. It’s free, it let’s people easily find, use, and contribute to your source code.
+
+Associate the GitHub repository with Tripal
+---------------------------------------------
+
+Once your module is on GitHub, consider joining the Tripal organization. Your lab group can exist as a team and maintain control over your code, but your projects will be listed in the main Tripal group.
+
+If you’d rather not, you can still tag your project as Tripal by clicking on the Manage Topics Link at the top of your repository.
+
+DOIs
+--------
+
+When your module is release ready, why not create a Digital Object Identifier (DOI) for it with `Zenodo <https://zenodo.org/>`_? It’s free! Sync your github account and create a new release (Zenodo won’t find old releases). You can then display your DOI badge on your module’s page.
+
+Additionally, there is a `Tripal Community group <https://zenodo.org/communities/tripal/>`_ on Zenodo. You can edit your record to associate your DOI with the Tripal community.
+
+Testing and Continuous Integration
+-----------------------------------
+
+`Tripal Test Suite <https://github.com/statonlab/TripalTestSuite>`_ is a full-featured testing module that makes writing tests much easier. which will automatically set up a PHPUnit and Travis testing environment for you.
+
+* Test with PHPUnit
+* Run tests as you push code with Travis CI
+
+
+Documentation
+--------------
+
+Every repository can include a README file that will be displayed on the repository page. A README file should at a minimum include:
+
+* An overview of the module
+* Instructions on how to install & use the module
+
+Consider documenting your Code itself. Tripal documents in the `Doxygen style <http://www.stack.nl/~dimitri/doxygen/>`_ which allows documentation webpages to be automatically generated. Even if you don’t build HTML documentation, the in-line code documentation will be very helpful to contributors.
+
+Coding Standards
+-----------------
+
+Drupal has defined `coding standards <https://www.drupal.org/docs/develop/standards/coding-standards>`_ that Tripal modules should meet.
+

docs/dev_guide/custom_data_loader.rst

@@ -0,0 +1,2 @@
+Creating Custom Data Loaders
+==============================

docs/dev_guide/custom_field.rst

@@ -0,0 +1,44 @@
+Creating a Custom Field
+========================
+
+The most common way that new content will be added to an existing site is by creating new fields, or field displays.  In Tripal v2 customizations were added by editing PHP templates files.  These template files were  relatively easy to create and customize, but they provided less flexibility and did not integrate well with other Drupal features such as GUI-based page layout and Drupal Views.  Tripal v3 fields now provide this flexibility.  They also support data exchange and data collections!
+
+By default Tripal v3 provides many fields for display of Chado data. However, you may find that these fields do not display data as you want, or you want to display data that the current fields do not already provide. This section of the Handbook describes how to create new fields that are integrated into the display, search and exchange abilities of both Drupal and Tripal.
+
+If you are already familiar with Drupal fields you may be aware of the API functions and hooks that Drupal provides.  However, for the quantity of fields needed to support biological data, the Drupal API hooks quickly become overwhelming.  Additionally, documentation for fields in the Drupal API can someitmes be difficult to discover when first working with fields.   Therefore, Tripal provides several new PHP classes to simplify creation of fields and to consolidate all functionality into one easy to find set of files.  To develop new fields you should be somewhat familar working with PHP's Object-Oriented Classes. The new classes provided by Tripal are these:
+
+
++----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| Class Name           | Description                                                                                                                                                                                                               |
++----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| TripalField          | The TripalField class provides the basic information about a new field. It provides loaders for extracting data from the database and functions for querying data managed by the field.                                   |
++----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| TripalFieldWidget    | The TripalFieldWidget class provides the necessary form elements when editing and Entity to allow the end-user to edit the value of the field (if desired). It provides the necessary validators and submittor functions. |
++----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| TripalFieldFormatter | The TripalFieldFormatter class provides the visualization of the field when viewed on the page.                                                                                                                           |
++----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| ChadoField           | The ChadoField class extends the TripalField class and provides the necessary settings to allow the field to map entities to data in Chado                                                                                |
++----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| ChadoFieldWidget     | Extends the TripalFieldWidget class but currently provides no additional functionality. Use this class when working with Chado data to ensure future backwards compatibility.                                             |
++----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| ChadoFieldFormatter  | Extends the TriplFieldFormatter class but currently provides no additional functionality. Use this class when working with Chado data to ensure future backwards compatibility.                                           |
++----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+
+
+
+The process for creating a custom field are as follows:
+
+* Determine the controlled vocabulary term that best describes the data your field will create.
+* Decide if you need the Chado field classes or the base Tripal field classes.  If you intend to work with data housed in Chado then you should use the Chado field classes.
+* Decide if you want to build your class manually from the ground up or speed development by using the Staton Lab Fields Generator tool.
+* Create new implementations of classes that extend those listed in the table above.  If you implement the functions properly your field is plug-and-play!  Tripal will find it and be able to use it.
+The rest of this section will walk you through these steps.
+
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Table of Contents
+
+   custom_field/select_vocab_terms
+   custom_field/manual_field_creation
+   custom_field/tripal_field_generator

docs/dev_guide/custom_field/manual_field_creation.rst

@@ -0,0 +1,2 @@
+Manual Field Creation
+======================

docs/dev_guide/custom_field/select_vocab_terms.rst

@@ -0,0 +1,2 @@
+Selecting Vocabulary Terms
+============================

docs/dev_guide/custom_field/tripal_field_generator.rst

@@ -0,0 +1,4 @@
+Easier Field Creation: Tripal Field Generator 
+============================================
+
+The Staton Lab has created a `specialized tool <https://github.com/statonlab/fields_generator>`_ for automatically generating a ChadoField or TripalField and populating them with the basic controlled vocabulary and database information.  Instructions are available at https://github.com/statonlab/fields_generator.

docs/dev_guide/custom_modules.rst

@@ -0,0 +1,6 @@
+Creating Custom Modules
+========================
+
+The Tripal API, in conjunction with the Drupal API allow developers to create their own modules that can "plug-in" with Tripal. Developers may create their own modules if they desire new or different functionality. A custom module can also be desired to house a sites "customizations." The Tripal API provides an interface for module developers to interact with the Chado database as well as the other Tripal core functions such as jobs management, materialized views, in-house vocabularies, external database cross-references, properties, etc. An understanding of the Drupal API and Drupal module development is required.
+
+Basic instructions for creating custom Drupal module can be found here:  https://www.drupal.org/docs/7/creating-custom-modules.   Before you can create custom modules that interact with Tripal, you should review those instructions and have practiced creating at least a simple module.   Tripal follows Drupal module development rules and API.  

docs/dev_guide/custom_web_services.rst

@@ -0,0 +1,2 @@
+Creating Custom Web Services
+==============================

docs/dev_guide/data_structures.rst

@@ -0,0 +1,59 @@
+Tripal Data Structures
+=======================
+
+This page explains the relationships between Entity types, Bundles (content type), Entities and Fields. These are data structures provided by Drupal that Tripal v3 uses to expose biological content through your Drupal site, and to provide flexibility in site customization.  It is important to understand these terms and their relationships before developing new modules or custom functionlaity.   A quick summary of these Drupal data structures are as follows:
+
+* Entity:  a discrete data record.  Entities are most commonly are seen as "pages" on a Drupal web site.
+* Field:  an "atomic" piece of ancillary data that describes, defines or expands the entity.  Common fields include the name of an entity, a "body" of descriptive text, etc.
+* Bundle:  a content type.  An entity must always have a content type.  Drupal provides several content types by default:  Basic Page and Article.  Tripal provides biological bundles (e.g. genes, organisms, etc).
+* Entity Type:  despite the confusing name, an entity type is simply a group of bundles that are somehow related.  Drupal provides a "Node" Entity type that includes the Page and Article bundles.  All of the Tripal bundles (content types) belong to the TripalEntity Entity type.
+
+
+The following figure describes the heirarchical relationship between Drupal Entity types (e.g. Node) in comparison with TripalEntity entity types (e.g. Chromosome, Germplasm, etc.). 
+
+
+.. figure:: /_images/dev_guide/data_structures/Terminology-Diagram.png
+   :scale: 100 %
+   :alt: Entity terminology guide
+
+
+Furthermore, fields are "attached" to a Bundle and hold unique values for each Entity. The following figure describes this relationship for a Gene Bundle that has several fields attached: name, description and organism.  Note that in this figure the Entity and each of the Fields are defined using a controlled vocabulary term.  As a result, bundles and fields can be described using the `Semantic Web <https://en.wikipedia.org/wiki/Semantic_Web>`_ concepts of a "subject", "predicate" and "object".  We will discuss more on controlled vocabularies a bit later in the Handbook.
+
+
+
+.. figure:: /_images/dev_guide/data_structures/Tripal-Semantic-web.png
+   :scale: 50 %
+   :alt: Semantic web
+   
+
+
+Bundles (Content Types)
+=======================
+
+Bundles are types of content in a Drupal site.  By default, Drupal provides the Basic Page and Article content types, and Drupal allows a site developer to create new content types on-the-fly using the administrative interface--no programming required.  Tripal also provides several Content Type by default. During installation of Tripal the Organism, Gene, Project, Analysis and other content types are created automatically.  The site developer can then create new content types for different biological data--again, without any programming required.
+
+In order to to assist with data exchange and use of common data formats, Tripal Bundles are defined using a controlled vocabulary term (cvterm). For example, a "Gene" Bundle is defined using the Sequence Ontology term for gene whose term accession is: SO:0000704. This mapping allows Tripal to compare content across Tripal sites, and expose data to computational tools that understand these vocabularies. You can create as many Tripal Content Types as you would like through Administration > Structure > Tripal Content Types provided you can define it using a controlled vocabulary term. 
+
+By default, Tripal uses Chado as its primary data storage back-end.  When a bundle is created, the Tripal Chado module allows you to map a Bundle to a table in Chado.  Thus, any content type desired can be define as well as how it is stored in Chado--all using the administrative interface.  
+
+Entity
+======
+
+An entity is a discrete data record.  Entities are most commonly seen as "pages" on a Drupal web site and are instances of a Bundle (i.e content type). When data is published on a Tripal site such as organisms, genes, germplasm, maps, etc., each record is represented by a single entity with an entity ID as its only attribute.   All other information that the entity provides is made available via Fields.  
+
+Fields
+=======
+
+A field is a reusable "data container" that is attached to a Bundle. Programmatically, each field provides one or more primitive data types, with validators and widgets for editing and formatters for display. Each field independently manages the data to which it assigned.  Just like with Bundles, Fields are also described using controlled vocabulary terms.  For example, a gene Bundle has a field attached that provides the name of the gene.   This field only provides the name and nothing more.  Tripal uses the `schema:name <http://schema.org/name>`_ vocabulary term to describe the field.  
+
+Field Instances
+================
+
+Fields describe "atomic" units of data that are associated with an entity.  For example, a "name" is an atomic unit of data about a Gene or Organism entity. Fields can be reused for multiple Bundles. For example, gene, mRNA, genetic markers and variants all have name data.  Despite that all of these Bundles provides a "name", we only need one field to describe that this data is a "name".  However, we may want to customize a field specific to each bundle.  Therefore, an Instance of a field is attached to a bundle, and field instances can then be customized differently.  The most important customization is the one that defines the Chado table from which the data for a field is retrieved.   Despite that field instances are attached to bundles, they become visible with Entities.  When an entity is loaded for display, Drupal examines all of the fields that are attached to the entity's bundle, and then populates the fields instances with data specific to the entity being loaded.
+
+Entity Types
+=============
+
+An entity type is simply a group of Bundles that have some similarity.  For examples Drupal provides a Node entity type. The Node entity type contains the Basic Page and Article Bundles.  Tripal v2 expanded the Node entity type when creating new content.  Tripal v3, however, uses an a new entity type named TripalEntity that provides the Organism, Gene, Analysis, etc. content types.  Using these new entity types provides a a more responsive solution then the Node entity type, is more flexible, and supports the new ontology-driven approach of Tripal v3.
+
+ 

docs/dev_guide/lorem_ipsum.rst

@@ -1,4 +0,0 @@
-Example file
-===============
-
-Lorem ipsum

docs/user_guide/learn_chado.rst

@@ -12,12 +12,12 @@ Learn Chado
 
 The primary data store for Tripal is Chado.  Chado is an open-source database schema managed by GMOD_.  Chado was selected for Tripal because it is open-source, it is maintained by the community in which anyone can provide input, and use of Chado encourages common data storage between online biological sites which decreases duplication of effort. Chado is meant to be installed into a PostgreSQL database and is designed to house a variety of biological data.  A good understanding of Chado is needed to take full advantage of Tripal.  For example, Tripal comes with a variety of content types. However, if you want to create new content types you must know how that data will be stored in Chado.  Additionally, use of the Bulk Loader (a tab-delimited data loader for custom data formats) requires a good understanding of Chado.  Finally, creating extensions to Tripal requires an understanding of Chado to write SQL and or new Tripal fields.  The following links provide training for Chado.
 
-===============
-Chado Resources
-===============
-`Chado Home Page <http://gmod.org/wiki/Chado>`_
-`Chado Tutorial <http://gmod.org/wiki/Main_Page>`_
-`Chado Table List <http://gmod.org/wiki/Chado_Tables>`_
-`Chado Best Practices <http://gmod.org/wiki/Chado_Best_Practices>`_
-===============
 
++---------------------------------------+
+|           Chado Resources             |
++---------------------------------------+
+| `Chado Home Page <http://gmod.org/wiki/Chado>`_ |
+|`Chado Tutorial <http://gmod.org/wiki/Main_Page>`_|
+|`Chado Table List <http://gmod.org/wiki/Chado_Tables>`_|
+|`Chado Best Practices <http://gmod.org/wiki/Chado_Best_Practices>`_|
++---------------------------------------+