The following section will take you through setting up your development environment to begin developing Native Apps for BaseSpace. If you are unfamiliar with Native Apps, please refer to the Native App Overview before proceeding.
There are two ways to develop Native applications:
All of these options share three packages, bsfs, spacedock, and docker. Both docker and spacedock are available via the apt-get repository and mono can also be found online.
Note: We recommend using at least Docker version 18.03.1. Issues may occur while uploading and downloading Docker images with earlier versions.
Important: Issues have been reported working with docker images for the most recent versions of Docker Desktop, 3.0 and above. Version 2.5.0.1 is known to work. If you experience unexpected EOF errors, we recoomend downgrading to 2.5.0.1 of Docker Desktop.
Our reccomendation is to use an amazon instance configured with these packages for testing, as it is the closest enviromenment to what we use for BaseSpace.
Please follow the instructions in this page for each setup.
What you will need for the following installation instructions:
Copy these files to your local computer
Ensure that you are connected to the internet on your local computer
Install VirtualBox
a. Select the VirtualBox installer for Mac
b. Click through the dialogs and install VirtualBox
c. Restart your computer
d. Find "Oracle VM VirtualBox" in your applications and open the program
In the VirtualBox window, select File -> Import Appliance...
Click Open Appliance...
Select the BaseSpace Native App VM.ova
Click Next
Ensure that the Reinitialize the MAC address of all selected network cards option is selected
Click Import, this will start importing the VM and the process may take several minutes
Verify that the Virtual Machine was successfully imported by selecting the new VM from the list in Virtual Box and clicking Start. A new window will open and begin loading your VM, if you are prompted to enter a password you have completed setting up your local dev environment.
On your local machine, open a terminal by launching the Terminal application
In your terminal's command line, please enter the following: ssh basespace@localhost -p2222 and hit Enter
You will be prompted for a password, the password is basespace
Install VirtualBox
a. Select the appropriate VirtualBox installer for your OS
b. Click through the dialogs and install VirtualBox
c. Restart your computer
d. Find "Oracle VM VirtualBox" in your applications and open the program
In the VirtualBox window, select File -> Import Appliance...
Click Open Appliance...
Select the BaseSpace Native App VM.ova
Click Next
Ensure that the Reinitialize the MAC address of all selected network cards option is selected
Click Import, this will start importing the VM and the process may take several minutes
Verify that the Virtual Machine was successfully imported by selecting the new VM from the list in Virtual Box and clicking Start. A new window will open and begin loading your VM, if you are prompted to enter a password you have completed setting up your local dev environment.
Download and install an SSH client if you do not already have one. Some examples of popular SSH clients: PuTTY, OpenSSH, and WinSCP
Open your SSH client, the following example will be using PuTTY
Fill in the following information:
Host Name (or IP Address): basespace@localhost Port: 2222
Click Open
A terminal will open and you will be prompted for a password, the password is basespace
Upon successful login to the Virtual Machine, you should see the terminal change to:
vagrant@precise64:~$
Congratulations, you've finished setting up your local computer as a development environment for your Native Application for BaseSpace!
You can now use the other documentation in the developer portal to begin building your Docker image for your Native App.
For more general information about native apps, please refer to the Native App Overview
For more information on how to build your Docker image for your application, please refer to Managing your Docker Image.
For more information about the conventions of Native Apps including automatic download, upload, authentication and error reporting, please refer to the Native App Conventions.
For information on how to debug and test your application locally, please refer to Testing and Debugging your Native App Locally
There are two ways to configure your machine to be a Native Apps Development environment.
Only 64-bit Linux operating systems are supported.
For this environment setup, we are using the following versions:
In your development machine, type
sudo apt-get update
Download the Docker repository key
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
Setup the Docker debian package
sudo sh -c "echo deb https://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
Update debian packages
sudo apt-get update
Install Docker 1.6.2 (currently supported with latest SpaceDock version)
sudo apt-get install lxc-docker-1.6.2
We'll have to enable communication over HTTP for your Docker service, this can be done by first opening the Docker config file:
sudo vim /etc/default/docker
Replace and uncomment DOCKER_OPTS with the following:
DOCKER_OPTS="-d -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock"
To run docker with the new changes, restart the docker service
sudo service docker restart
For Ubuntu 15, in order to have the Docker service start on machine startupm run the following command:
systemctl enable docker
Download the SpaceDock docker image
sudo docker pull basespace/spacedock
This Docker image already contains BSFS, Mono, and SpaceDock.
Now, we will have to run this SpaceDock Docker image with the following command
Use --follow to see the log as its running
sudo docker run -d --net=host --privileged=true -v /data:/data -v /genomes:/genomes basespace/spacedock spacedock -a <agent_id> -m <mission_control_uri>
This command will return a container_id which is the docker container id for the spacedock docker image that you just ran.
You can see what is going on within the spacedock docker container (to view jobs as they are processing) by using the container_id from above:
sudo docker attach <container_id>
The machine is now ready to receive Native apps jobs from BaseSpace.
Any Linux machine can be configured as a Native Apps development machine by simply installing the following packages:
To make things easier, we would recommend using sudo su to work as the root user on your machine, this will make the following commands simpler. If there is a permissions issue with any of the requests, you will need to use root access by either doing the above or adding sudo appropriately in each command.
In your development machine, type
sudo apt-get update
Download the Docker repository key
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
Setup the Docker debian package
sudo sh -c "echo deb https://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
Update debian packages
sudo apt-get update
Install Docker 1.6.2 (currently supported with latest SpaceDock version)
sudo apt-get install lxc-docker-1.6.2
We'll have to enable communication over HTTP for your Docker service, this can be done by first opening the Docker config file:
sudo vim /etc/default/docker
Replace and uncomment DOCKER_OPTS with the following:
DOCKER_OPTS="-d -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock"
To run docker with the new changes, restart the docker service
sudo service docker restart
For Ubuntu 15, in order to have the Docker service start on machine startupm run the following command:
systemctl enable docker
Docker is now set up in your environment!
Add the mono debian key
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
Specify the mono version that you would like to install, for Native App development we are currently using mono version 3.12.0
echo "deb http://download.mono-project.com/repo/debian wheezy/snapshots/3.12.0 main" | sudo tee /etc/apt/sources.list.d/mono-xamarin.list
Update debian package repositories
sudo apt-get update
Install the specified mono version
sudo apt-get install -y mono-complete
Now you should have mono properly setup and ready to go!
Add the SpaceDock debian repository
echo deb http://basespace-apt.s3.amazonaws.com spacedock main | sudo tee -a /etc/apt/sources.list; sudo apt-get update; sudo apt-cache policy spacedock
Update debian packages
sudo apt-get update
Install SpaceDock
sudo apt-get install spacedock
SpaceDock should now be ready to use in your Native Apps development environment!
Add BSFS debian repository
sudo add-apt-repository "deb http://basespace-apt.s3.amazonaws.com bsfs main"
sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
Update and install the BSFS package
sudo apt-get update
sudo apt-cache policy bsfs
sudo apt-get -y --force-yes install bsfs
Note: In some cases, your 64-bit OS may not be detected. In this case, you will see an error when you try apt-get update so to fix this you will have to open the file /etc/apt/sources.list and add [arch=amd64] to the bsfs repository so it looks like deb [arch=amd64] http://basespace-apt.s3.amazonaws.com bsfs main.
In the root folder on your machine, make the /data and /genomes folders. These are necessary in order for BaseSpace to run the app.
sudo mkdir /data
sudo mkdir /genomes
Before your machine can receive jobs from BaseSpace, you will have to run SpaceDock using the following command
sudo spacedock -a <agent_id> -m <mission_control_url>
Now you are ready to build and test your BaseSpace Native app!
There are a few practices that will help clean up the development environment. As you run your Native apps through the development environment, the data can remain on the machine after the job is executed. In order to clean up the host development machine, we would recommend:
/data directory
/data directory on the host machine