Feature Overview

Owned by Craig Willis
Last updated: Feb 15, 2017

The Labs Workbench is currently in beta testing. Documentation is subject to change.

This document refers to the Labs Workbench demonstration instance available at https://www.workbench.nationaldataservice.org.

Overview

What is NDS Labs Workbench?

Current version

Version 1.0.8 was deployed on February 12, 2017.

Features

  • Catalog of over 40 data management and analysis tools 
  • Support for user-defined applications, including the ability to import and export application specifications
  • Integrated file manager
  • Access to console and logs for every running application
  • Persistent per-account "home" directory accessible from every running container
  • Per-account resource constraints and quotas
  • Multi-node Kubernetes (v1.5) cluster hosted on NCSA Nebula (OpenStack) with support for:
    • Gluster shared filesystem
    • Monitoring (Nagios)
    • Backup (disaster recovery)
  • Optional account registration and approval workflow to support workshops
  • Allow admin user to shut down user services, in case of emergency
  • Cloud-based development environments for popular languages including C/C++, Go, Java, NodeJS, PHP, Python
  • Integrated Docker build/tag/push support for deploying new services
  • Remote access to TCP services via SSH tunneling

Demonstration videos

The following videos do not reflect the current state of the system

Registration and login

This brief video demonstrates the registration, approval, and login workflow.  

Key features:

  • Email verification
  • Registered accounts must be approved by NDS Labs staff.  
  • Each account is allocated resources limited by # of CPUs, memory, and storage.
  • Password recovery

Browse and manage applications

This video demonstrates basic browsing and running of applications in NDS Labs.

Key features:

  • Filter-based searching of application catalog
  • Applications are added with reasonable default configurations
  • Advanced configuration of each application component is possible
  • "Home" directory mounted on all containers allows users to copy and retain data
  • Integrated file manager with upload functionality

Create user-defined applications

This video demonstrates creating and running a user-defined application.

Key features:

  • Ability to add, edit, clone, and delete user specifications
  • Links to detailed information page on NDS wiki

Development environment - simple example

This video demonstrates a simple development scenario using a PHP development environment.

Key features:

  • Development environment with interactive debugging

Docker build/tag/push

A challenge of running services in a container based environment is allowing users to be able to build, run, and push new containers from within a running container.  This is achieved by using the "Docker-in-Docker" paradigm (https://hub.docker.com/_/docker/).

Key features:

  • Allow users to build, tag, and push Docker images

Development environment - remote access

While the Xpra or VNC-based interfaces are convenient, some developers will prefer to run local IDEs while accessing services running in NDS Labs.  In this video, we demonstrate using the Chisel SSL-over-HTTP tunneling service to connect a local PyCharm instance to an NDS Labs-hosted Clowder stack:

  • Local development
  • Remote access to TCP services via

Development environment - Clowder extractor development

This video demonstrates a more complex development scenario. In this example, an application-specific development environment has been provided for Clowder extractor development based on the PyCharms IDE. The PyCharms IDE is a desktop application, so it is exposed in NDS Labs using Xpra.

Key features:

  • Application-specific development container. This container has all of the dependencies required for pyClowder development.
  • Desktop IDE containerized via Xpra (X-forwarding)
  • Interactive Python debugging with running Clowder application stack