The Labs Workbench is currently in beta testing. Documentation is subject to change. |
This document refers to the Labs Workbench demonstration instance available at https://www.workbench.nationaldataservice.org.
This guide is for users of the Labs Workbench that are interested in:
Preparing your application for use in Labs Workbench requires the following:
The Labs Workbench requires that all services be packaged as Docker containers.
If you are unfamiliar with Docker, start with one of these tutorials:
Note that you can use the Docker service provided in the Labs Workbench if you don't have access to Docker locally.
A few things to keep in mind:
Often our applications depend on other services. A particular web application might consist of a web server, application server, database, and search engine. Dependencies introduce complexity when running containers. For example:
Labs Workbench supports dependencies through the "dependencies" array in the service specification. Dependencies are either required or optional. If required, dependencies are always started before the dependent services.
For all services associated with a single application, Labs Workbench injects environment variables into the containers during startup. These environment variables follow the Docker compose model of "SERVICEKEY_PORT_PORTNUM_TCP_ADDR" and "SERVICEKEY_PORT_PORTNUM_TCP_PORT". For example, if your service depends on MongoDB (key=mongo), your container will have the following environment variables at runtime:
$ env | grep ^MONGO MONGO_PORT_27017_TCP_ADDR=<internal IP> MONGO_PORT_27017_TCP_PORT=27017 |
Your application can always rely on these environment variables to be present.
Labs Workbench supports the following types of environment variables:
Variable | Description |
---|---|
NDSLABS_DOMAIN | Domain of the Workbench cluster |
NDSLABS_HOSTNAME | Full host name of your running service |
NDSLABS_STACK | Unique ID of your application stack |
NDSLABS_HOME | Home directory for your account |
NDSLABS_EMAIL | Account email address |
Labs Workbench supports custom configuration through the use of environment variables.
Currently, containers always run as root. However, because some containers use non-root users, permissions on your account filesystem are permissive.
TODO: Protocols, port numbers, access type, and context path
TODO: Service volumes versus customization
Labs Workbench relies on Docker logging. The Docker logging driver reads log events from stderr and stdout only. If possible, all application log messages should be sent to stdout or stderr. If your application logs to a file, you will not be able to view it through the Labs Workbench log viewer, but you can always use the console feature to access it.
In a future release, the Workbench will provide usage information for each container to help estimate CPU and memory requirements. In the meantime, the best solution is to run your container outside of the Workbench or in the provided Docker environment and use the "docker stats" command.
$ docker stats CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O 201337f8e5c7 0.00% 1.987 MB / 8.376 GB 0.02% 385.2 kB / 1.778 MB 0 B / 0 B |
This information can be used to estimate service resource requirements, including min/max CPU and min/max memory. Memory estimates are more important at this time. Most services will likely work with the default CPU limits.
Labs Workbench uses Kubernetes internally for container orchestration. While you do not need to understand Kubernetes to use or develop for Labs Workbench, some concepts might useful for those interested:
To submit to the official NDS Labs catalog, your image must:
You can actually set up Docker Hub to automatically build your images when changes occur to the underlying source.
We recommend setting up such automated builds for any custom Docker images referenced by your spec(s).
A fantastic guide for doing so can be found here: https://docs.docker.com/docker-hub/builds/
You simply need to:
While extremely convenient for asynchronous development and testing, Docker Hub's automated builds are by no means perfect.
if you desire a more rapid "code-build-test-repeat" workflow, you may want to stick to manual builds until you reach a stable state as speed seems to be the main problem with automated builds.
Some things that contribute to the slow build times:
You will likely want to stick with this pattern of semantic versioning that has worked well for other projects.
Below is a set of build rules that can help keep your images up-to-date automatically and versioned appropriately.
New commits pushed to master on GitHub should be built and pushed to latest on Docker Hub.
This way, latest will always reflect your latest pushed code on the master branch.
New tags pushed to GitHub (i.e. 1.0.0) should be built and pushed to the same name on Docker Hub.
Benefits of proper versioning:
In addition, you may want every new tag pushed to be built and pushed to stable.
This way, the stable tag will always be your latest tagged release on GitHub.
Our official application specs for Labs Workbench are housed here: https://github.com/nds-org/ndslabs-specs
The following snippet can help you add your service to the Workbench's official catalog:
This will create a personal copy of the repo that you can clone and modify as you see fit:
See https://help.github.com/articles/fork-a-repo/ for more information
export gitUsername=YOUR_GIT_USERNAME_HERE git clone https://github.com/${gitUsername}/ndslabs-specs cd ndslabs-specs/ |
export specKey=YOUR_SPEC_KEY_HERE git checkout -b ${specKey} |
For each application (i.e. entries in the catalog that users can add), create a new folder and place your spec(s) there.
mkdir ${specKey} cd ${specKey}/ vi ${specKey}.json # Paste contents of JSON spec into this file git add ${specKey}.json |
Repeat this step for all dependent services that users will need to run this application.
git commit -a -m "Added new application: ${specKey}" git push origin |
This will create a request to merge the changes from your branch back into the master branch of the official Labs Workbench catalog:
Navigate to https://github.com/YOUR_GIT_USERNAME_HERE/ndslabs-specs
Click on New pull request at the top-left of the file table
Click the compare across forks link at the top-right. This should display some new drop-down menus in the section directly below it.
In the box on the left, choose:
base fork: nds-org/ndslabs-specs
base branch: master
See https://help.github.com/articles/creating-a-pull-request/ for more information