Windows 10 Linux subsystem for Python developers

front_logo.png

Tabula rasa

One of the hardest days in teaching introduction to bioinformatics material is the first day: Setting up your machine. This is difficult for one very simple reason, everyone tends to have a different one. While I have seen a very large bias towards Macs in academia, there are plenty of people that keep their Windows machines as a badge of pride...me included.

As a Python developer in an academic environment, I am a huge proponent of Anaconda builds. Anaconda has taken very large strides in making a seemingly ubiquitous environment for all platforms. However, because of the enrichment of Macs in the academic field and most computing clusters being Unix-like systems, it is difficult to easily translate the platform differences at a finer grain level.

To ease the headaches of those users, this first blog piece is how I set up my Windows 10 system for bioinformatic development.


The Setup

  • Windows 10 : version 1709 or later
  • Internet connection
  • Text editor (Notepad++ or VScode)

Step 1: The Windows Subsystem for Linux

I am going to be suggesting a lot of keyboard shortcuts during this tutorial, so I figured I would list them here.

Win: Windows key
Search: Win + s
Run: Win + r

  • The first thing we need to do is make sure your Windows is up to date. Run winver and make sure the version is greater than or equal to 1709

winver

A new window should pop-up that looks like this:

winver_output

  • NOTE: If your version is 1709 or higher, we can keep going.

Search for 'Turn Windows features'

feature search

  • ☑ Windows Subsystem for Linux

wsl check

  • Restart your machine (Win + x, then u + u)

Step 2: Installing Ubuntu

  • Search for 'Microsoft Store'
  • Look for 'Ubuntu' (or whatever Linux build you are interested in) when the store opens and click on the first option that shows up

win store

  • Press 'Get'

store get

  • Follow all prompts

As a developer, I am regularly opening new terminals. Therefore, I don't want to search in the Start menu for it every time. Thankfully, Windows 10 has a very strong keyboard shortcut system. To leverage it, do the following:

  1. Search for Ubuntu (or your selected build)
  2. Right click on the result and select 'Pin to Taskbar' (shown below)
  3. Then move the pin to the far left, just next to the Windows Logo

pin

taskbar

With this in place, the user can now use Win + 1 to open a terminal. All Win + 1 does is open the first program pinned to the taskbar.

Search for Ubuntu (or your Linux build), and run it once. It will say 'Installing, this may take a few minutes...'. This step needs to be done for the whole file system to be setup (takes a couple of minutes). Once it is done and it asks you for a Username and password, you can choose to close it (if you would like to be root by default), or fill it out (if you don't want to be root by default).


Step 3: Installing an X Server

The Linux subsystem doesn't natively allow the creation of graphical windows. We can fix that by installing an X server on Windows that will catch any X11 requests and render them. Many people will choose to use Xming, but I prefer VcXsrv because it has better clipboard support between Windows and Linux. It is up to you though. However, from here on out, the guide will assume you installed VcXsrv. Nothing changes between the two as far as setup, just the name.

Note: X11 is just a command system for generating windows

  • Whichever X server you choose, install it.

Like the Linux terminal, you may want to add a layer of convenience when it comes to the X server. The best way to do that is to set it to start up on boot. Do the following to do that:

  • Search for VcXsrv and right click to select 'Open file location'

file loc

  • When the window that opens, right click the icon labeled "VcXsrv" and select 'Copy Link'

copy link

  • Run shell:startup to open up another window. This brings you to the folder that loads certain programs at startup. Paste the copied VcXsrv into this new window.

Step 4: File Convenience

If you haven't done a lot of stuff on your computer, you may need to change some settings to make things a little easier. To make hidden files visible and to show file extensions, do the following:

  • Win + e to open a file explorer
  • Click on the View tab
  • check both 'File name extensions' and 'Hidden items'

extensions


Step 5: Initializing the Environment

If this is your own computer, it can get very tedious entering your password every time you want to install something in the Ubuntu shell. Therefore, I strongly encourage you to Run CMD, and paste the following command:

C:\"Program Files"\WindowsApps\CanonicalGroupLimited.UbuntuonWindows_1804.2018.817.0_x64__79rhkp1fndgsc\ubuntu.exe config --default-user root

This makes your terminal sessions start as root user by default. No need to use any passwords anymore.


Now, below is a series of commands that you can just copy and paste to set up your Ubuntu shell. This isn't very interesting, but each command has a comment explaining the reasoning of the commands:

apt-get update # this updates makes sure your source list is up-to-date

apt-get upgrade # this command upgrades all installed packages

apt-get install firefox # Installs firefox browser, which can be used for jupyter notebooks/lab


Many people have probably already installed Anaconda on Windows, and may think this next step is unnecessary. However, if you plan on running any Python projects from the Ubuntu shell, it is important that you install a Python distribution on Linux. The reason is that the Windows build of Ananconda is for Windows. It will not (and cannot) be used by Linux. Therefore:

  • Choose one of the following:
    1. Miniconda:

      wget -c https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -P ~/Downloads

      This downloads the latest Miniconda build. Miniconda is a lightweight version of the Anaconda Python build. It only contains Python 3.x and the conda package manager. Since this is our root environment, we don't want to bloat it with a bunch of libraries we may or may not use.

    2. Anaconda:

      wget -c https://repo.anaconda.com/archive/Anaconda3-5.2.0-Linux-x86_64.sh -P ~/Downloads

      This will download the full Anaconda build of Python. This is a sizable install, so make sure you have the space for it.

  • bash ./Downloads/<name of the file> and follow the prompts

    For example: bash ./Downloads/Miniconda3-latest-Linux-x86_64.sh

    I suggest you just use the defaults for all the prompts and have it install in your home directory.

conda update conda # The ensures that conda is up to date


The following will create and modify the .condarc file, which controls the behavior of conda.

conda config --add channels defaults # the main anaconda channel for package downloads

conda config --add channels anaconda # Alias channel for defaults

conda config --add channels conda-forge # 3rd party created/maintained packages

conda config --add channels bioconda # Bioconda is a channel for the conda package manager specializing in bioinformatics software.

conda config --set always_yes True # Makes it so you don't have to type y when you want to install something with conda


Step 6: .bashrc Setup

Now to make sure our Linux subsystem can communicate with our Windows desktop, we have to add some things to our home directory.

The first step here will be to modify our .bashrc file. The rc of .bashrc stands for 'Run commands'. What this means is that when a terminal is opened, these commands are run before the user does anything. This is helpful when adding or modifying environmental variables.

We already installed the Windows X server (VcXsrv). What that did was create a server that is listening for X11 commands coming from the Linux terminal. To make that communication happen, we need to add a line to .bashrc

Paste the following command in the terminal:

echo "export DISPLAY=:0.0" >> ~/.bashrc && source ~/.bashrc

This command adds the display forwarding address to .bashrc and makes the changes happen in the current shell


Additionally, there are some quality of life aliases that can be added to .bashrc:

echo "alias ll='ls -lh --color=auto'" >> ~/.bashrc # Long form list of directory contents, colorized, and human-readable

echo "alias ls='ls -h --color=auto'" >> ~/.bashrc # Human-readable and colorized listing of directory contents

echo "alias grep='grep --color=auto'" >> ~/.bashrc # Colorizes grep results

echo "alias update='sudo apt-get update && sudo apt-get upgrade'" >> ~/.bashrc # Adds a command that will update and upgrade all packages at the same time. This is good to do from time to time.

source ~/.bashrc # starts the changes to .bashrc in current terminal


Linux has full access to your Windows file system. However, it can be a little disorienting to find your way around. This is because Windows users are used to starting at the Desktop. To see your drives in the Ubuntu shell, they are found at:

ls /mnt

mnt

Due to how illogical the system can be, here are a couple of commands to copy & paste into the Ubuntu shell that would make things so much easier.

ln -s /mnt/<drive letter> ~/<drive letter> # replace the < ... > with the drive letter name. For example:

ln -s /mnt/c ~/c

Depending on what your drive letters are, or how many you have, this command will make a 'shortcut' to those drives in your Ubuntu home directory. With this, we can start navigating through our Windows files like this:

cd c/Users/Public/Documents

Just as Linux has full access to the Windows file system, Windows has full access to Linux. However, where the Ubuntu home directory is stored (relative to the Windows file system) is somewhat confusing. For the sake of brevity, I will give you the path (if you installed Ubuntu through the store).

If you are root:

C:\Users\<username>\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\root

Where <username> is your Windows username.

If you are not root:

C:\Users\<username>\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\home\<linux_username>

Where <username> is your Windows user name, and <linux_username> is the Linux username that you setup when you first opened Ubunut.

For example, mine would look like: C:\Users\<username>\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\root\

I suggest you Win + e to open up a File Explorer, and paste: C:\Users\<username>\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\ (one folder above your home directory) into the address bar and right click the name of your home directory, and select 'Pin to Quick access' (seen below).

qaccess


Step 8: Restart and ready to roll

My next blog will be covering conda env from the viewpoint of a bioinformaticist. This will introduce the ideas of why virtual environments are helpful, why you should use them, and basic project structures.

I hope you found this blog helpful. Feel free to post a comment below regarding critiques/criticisms, edits, and/or thanks. Look forward to hearing from you.

Additionally, I have posted examples of the config files that I use personally at https://github.com/betteridiot/blog. Please use & abuse them as you see fit. If you want to use them as is, just copy them into your home directory (in your linux terminal).


Name *
Name