Clowder Examples

A Simple Clowder Example

A user/developer wishes to deploy an instance of Clowder.

User creates a new project and names it "Basic Clowder Deployment"
User drags clowder service to project canvas
User drags mongodb to canvas as database backend for Clowder
User drags elasticsearch to canvas as means of indexing text data for Clowder
User drags rabbitmq as the distributed messaging bus used by Clowder to trigger extraction services
User drags extractor-1 to canvas as extraction service that will be available for uploaded files/datasets
User drags extractor-2 to canvas as extraction service that will be available for uploaded files/datasets
User drags a connection between mongodb and clowder
User drags a connection between elasticsearch and clowder
User drags a connection between rabbitmq and clowder
User drags a connection between extractor-1 and rabbitmq
User drags a connection between extractor-2 and rabbitmq

User clicks "Commit" button to deploy the project on NDS Labs cloud resources (e.g. SCSC Cloud, TACC Rodeo, NCSA Nebula). The user does not need to specify which resources as this will be assigned automatically based on automatically determined requirements (e.g. needed computation, storage, proximity to data, etc...)
A status page is returned providing a list of the instanciated services, virtual machines, their status (start time/running/idle), details (e.g. cores, memory), and URL's to the stood up services (e.g. Clowder)
User can now go to/share Clowder URL and use/experiment with the service

A More Involved Clowder Example

A user/developer wishes to deploy an instance of Clowder.

User creates a new project and names it "Clowder Deployment"
User drags clowder service to project canvas
User drags mongodb to canvas as database backend for Clowder
User drags elasticsearch to canvas as means of indexing text data for Clowder
User drags irods to canvas as an alternative storage option for uploaded data to Clowder
User drags nginx as means of load balancing Clowder requests
User drags rabbitmq as the distributed messaging bus used by Clowder to trigger extraction services
User drags extractor-1 to canvas as extraction service that will be available for uploaded files/datasets
User drags extractor-2 to canvas as extraction service that will be available for uploaded files/datasets
User drags extractor-3 to canvas as extraction service that will be available for uploaded files/datasets
User drags a connection between mongodb and clowder
User drags a connection between elasticsearch and clowder
User drags a connection between irods and clowder
User drags a connection between nginx and clowder
User drags a connection between rabbitmq and clowder
User drags a connection between extractor-1 and rabbitmq
User drags a connection between extractor-2 and rabbitmq
User drags a connection between extractor-3 and rabbitmq

User clicks "Commit" button to deploy the project on NDS Labs cloud resources
A status page is returned providing a list of the instanciated services, virtual machines, their status, details, and URL's to the stood up services (e.g. Clowder)
User can now go to/share Clowder URL and use/experiment with the service

ARPAE TERRA Project Example

User/developer wishes to deploy a data resource along the lines of that used by the APRAE TERRA project.

User creates a new project and names it "TERRA Deployment"
User drags clowder service to project canvas
User drags mongodb to canvas as database backend for Clowder
User drags elasticsearch to canvas as means of indexing text data for Clowder
User drags rabbitmq as the distributed messaging bus used by Clowder to trigger extraction services
User drags plantcv extractor to canvas as extraction service that will be available for uploaded files/datasets
User drags jupyter lab notebook to canvas as means of working with data stored in Clowder
User drags a connection between mongodb and clowder
User drags a connection between elasticsearch and clowder
User drags a connection between rabbitmq and clowder
User drags a connection between plantcv extractor and rabbitmq
User drags a connection between jupyter and clowder
User drags a connection between jupyter and plantcv extractor allowing user to call/re-run this code within Jupyter

User clicks "Commit" button to deploy the project on NDS Labs cloud resources
A status page is returned providing a list of the instanciated services, virtual machines, their status, details, and URL's to the stood up services (e.g. Clowder, Jupyter Lab Notebook)
User can now go to/share Clowder URL and use/experiment with the service
User can now go to/share Jupyter URL and user/experiment with any data contained within the Clowder instance

XSEDE ECS: Video Analytics Tableau Example

User/developer wishes to deploy a data resource along the lines of that used by the XSEDE VAT project (http://vat.xsede.org/) to store and analyze videos.

User creates a new project and names it "VAT Deployment"
User drags clowder service to project canvas
User drags mongodb to canvas as database backend for Clowder
User drags elasticsearch to canvas as means of indexing text data for Clowder
User drags rabbitmq as the distributed messaging bus used by Clowder to trigger extraction services
User drags the shotextractor to canvas as extraction service that will be available for uploaded video files in order to segment into distinct shots
User drags a connection between mongodb and clowder
User drags a connection between elasticsearch and clowder
User drags a connection between rabbitmq and clowder
User drags a connection between shotextractor and rabbitmq

User clicks "Commit" button to deploy the project on NDS Labs cloud resources
A status page is returned providing a list of the instanciated services, virtual machines, their status, details, and URL's to the stood up services (e.g. Clowder)
User can now go to/share Clowder URL and use/experiment with the service