This document refers to the Labs Workbench demonstration instance available at https://www.workbench.nationaldataservice.org.

Overview

What is NDS Labs Workbench?

Current version

Version 1.0.8 was deployed on February 12, 2017.

Features

• Catalog of over 40 data management and analysis tools 
• Support for user-defined applications, including the ability to import and export application specifications
• Integrated file manager
• Access to console and logs for every running application
• Persistent per-account "home" directory accessible from every running container
• Per-account resource constraints and quotas
• Multi-node Kubernetes (v1.5) cluster hosted on NCSA Nebula (OpenStack) with support for:
  
  • Gluster shared filesystem
  • Monitoring (Nagios)
  • Backup (disaster recovery)

• Optional account registration and approval workflow to support workshops

• Allow admin user to shut down user services, in case of emergency
• Cloud-based development environments for popular languages including C/C++, Go, Java, NodeJS, PHP, Python
• Integrated Docker build/tag/push support for deploying new services
• Remote access to TCP services via SSH tunneling
  
  

Demonstration videos

Registration and login

This brief video demonstrates the registration, approval, and login workflow.  

Key features:

• Email verification
• Registered accounts must be approved by NDS Labs staff.  
• Each account is allocated resources limited by # of CPUs, memory, and storage.
• Password recovery

Browse and manage applications

This video demonstrates basic browsing and running of applications in NDS Labs.

Key features:

• Filter-based searching of application catalog
• Applications are added with reasonable default configurations
• Advanced configuration of each application component is possible
• "Home" directory mounted on all containers allows users to copy and retain data
• Integrated file manager with upload functionality

Create user-defined applications

This video demonstrates creating and running a user-defined application.

Key features:

• Ability to add, edit, clone, and delete user specifications
• Links to detailed information page on NDS wiki

Development environment - simple example

This video demonstrates a simple development scenario using a PHP development environment.

Key features:

• Development environment with interactive debugging

Docker build/tag/push

A challenge of running services in a container based environment is allowing users to be able to build, run, and push new containers from within a running container.  This is achieved by using the "Docker-in-Docker" paradigm (https://hub.docker.com/_/docker/).

Key features:

• Allow users to build, tag, and push Docker images

Development environment - remote access

While the Xpra or VNC-based interfaces are convenient, some developers will prefer to run local IDEs while accessing services running in NDS Labs.  In this video, we demonstrate using the Chisel SSL-over-HTTP tunneling service to connect a local PyCharm instance to an NDS Labs-hosted Clowder stack:

• Local development
• Remote access to TCP services via

Development environment - Clowder extractor development

This video demonstrates a more complex development scenario. In this example, an application-specific development environment has been provided for Clowder extractor development based on the PyCharms IDE. The PyCharms IDE is a desktop application, so it is exposed in NDS Labs using Xpra.

Key features:

• Application-specific development container. This container has all of the dependencies required for pyClowder development.
• Desktop IDE containerized via Xpra (X-forwarding)
• Interactive Python debugging with running Clowder application stack