Windows 10 Linux subsystem for Python developers
One of the hardest days in teaching introduction to bioinformatics material is the first day: Setting up your machine. This is difficult for one very simple reason, everyone tends to have a different one. While I have seen a very large bias towards Macs in academia, there are plenty of people that keep their Windows machines as a badge of pride...me included.
As a Python developer in an academic environment, I am a huge proponent of Anaconda builds. Anaconda has taken very large strides in making a seemingly ubiquitous environment for all platforms. However, because of the enrichment of Macs in the academic field and most computing clusters being Unix-like systems, it is difficult to easily translate the platform differences at a finer grain level.
To ease the headaches of those users, this first blog piece is how I set up my Windows 10 system for bioinformatic development.
As of the May 2019 update to Windows 10 (v 1903), Ubuntu comes standard with Windows.
One of the biggest benefits of this change is file handling. Previously, Windows could only modify files on the Ubuntu file system, but not create. Now, by using
\\wsl$\Ubuntu in the address bar of your file explorer, one can directly access any Linux-based file, create additional ones, or remove them and the changes will be seen on the Linux-side as well.
Step 1: The Windows Subsystem for Linux
I am going to be suggesting a lot of keyboard shortcuts during this tutorial, so I figured I would list them here.
Win: Windows key
- The first thing we need to do is make sure your Windows is up to date. Run
winverand make sure the version is greater than or equal to 1709
A new window should pop-up that looks like this:
- NOTE: If your version is 1709 or higher, we can keep going.
Search for 'Turn Windows features'
- ☑ Windows Subsystem for Linux
- Restart your machine (
Step 2: Installing Ubuntu
- Search for 'Microsoft Store'
- Look for 'Ubuntu' (or whatever Linux build you are interested in) when the store opens and click on the first option that shows up
- Press 'Get'
- Follow all prompts
As a developer, I am regularly opening new terminals. Therefore, I don't want to search in the Start menu for it every time. Thankfully, Windows 10 has a very strong keyboard shortcut system. To leverage it, do the following:
- Search for Ubuntu (or your selected build)
- Right click on the result and select 'Pin to Taskbar' (shown below)
- Then move the pin to the far left, just next to the Windows Logo
With this in place, the user can now use
1 to open a terminal. All
1 does is open the first program pinned to the taskbar.
Search for Ubuntu (or your Linux build), and run it once. It will say 'Installing, this may take a few minutes...'. This step needs to be done for the whole file system to be setup (takes a couple of minutes). Once it is done and it asks you for a Username and password, you can choose to close it (if you would like to be root by default), or fill it out (if you don't want to be root by default).
Step 3: Installing an X Server
The Linux subsystem doesn't natively allow the creation of graphical windows. We can fix that by installing an X server on Windows that will catch any X11 requests and render them. Many people will choose to use Xming, but I prefer VcXsrv because it has better clipboard support between Windows and Linux. It is up to you though. However, from here on out, the guide will assume you installed VcXsrv. Nothing changes between the two as far as setup, just the name.
Note: X11 is just a command system for generating windows
- Whichever X server you choose, install it.
Like the Linux terminal, you may want to add a layer of convenience when it comes to the X server. The best way to do that is to set it to start up on boot. Do the following to do that:
- Search for VcXsrv and right click to select 'Open file location'
- When the window that opens, right click the icon labeled "VcXsrv" and select 'Copy Link'
shell:startupto open up another window. This brings you to the folder that loads certain programs at startup. Paste the copied VcXsrv into this new window.
Step 4: File Convenience
If you haven't done a lot of stuff on your computer, you may need to change some settings to make things a little easier. To make hidden files visible and to show file extensions, do the following:
eto open a file explorer
- Click on the View tab
- check both 'File name extensions' and 'Hidden items'
Step 5: Initializing the Environment
If this is your own computer, it can get very tedious entering your password every time you want to install something in the Ubuntu shell. Therefore, I strongly encourage you to Run CMD, and paste the following command:
C:\"Program Files"\WindowsApps\CanonicalGroupLimited.UbuntuonWindows_1804.2018.817.0_x64__79rhkp1fndgsc\ubuntu.exe config --default-user root
This makes your terminal sessions start as root user by default. No need to use any passwords anymore.
Now, below is a series of commands that you can just copy and paste to set up your Ubuntu shell. This isn't very interesting, but each command has a comment explaining the reasoning of the commands:
> apt-get update> # this updates makes sure your source list is up-to-date
> apt-get upgrade> # this command upgrades all installed packages
> apt-get install firefox> # Installs firefox browser, which can be used for jupyter notebooks/lab
Many people have probably already installed Anaconda on Windows, and may think this next step is unnecessary. However, if you plan on running any Python projects from the Ubuntu shell, it is important that you install a Python distribution on Linux. The reason is that the Windows build of Ananconda is for Windows. It will not (and cannot) be used by Linux. Therefore:
- Choose one of the following:
wget -c https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -P ~/Downloads
This downloads the latest Miniconda build. Miniconda is a lightweight version of the Anaconda Python build. It only contains Python 3.x and the
conda package manager. Since this is our root environment, we don't want to bloat it with a bunch of libraries we may or may not use.
wget -c https://repo.anaconda.com/archive/Anaconda3-5.2.0-Linux-x86_64.sh -P ~/Downloads
This will download the full Anaconda build of Python. This is a sizable install, so make sure you have the space for it.*
bash ./Downloads/<name of the file>and follow the prompts
I suggest you just use the defaults for all the prompts and have it install in your home directory.
> conda update conda> # The ensures that
> conda> is up to date
The following will create and modify the
.condarc file, which controls the behavior of
> conda config --add channels defaults> # the main anaconda channel for package downloads
> conda config --add channels anaconda> # Alias channel for
> conda config --add channels conda-forge> # 3rd party created/maintained packages
> conda config --add channels bioconda> # Bioconda is a channel for the
> conda> package manager specializing in bioinformatics software.
> conda config --set always_yes True> # Makes it so you don't have to type
> y> when you want to install something with
Now to make sure our Linux subsystem can communicate with our Windows desktop, we have to add some things to our home directory.
The first step here will be to modify our
.bashrc file. The
.bashrc stands for 'Run commands'. What this means is that when a terminal is opened, these commands are run before the user does anything. This is helpful when adding or modifying environmental variables.
We already installed the Windows X server (VcXsrv). What that did was create a server that is listening for X11 commands coming from the Linux terminal. To make that communication happen, we need to add a line to
Paste the following command in the terminal:
echo "export DISPLAY=:0.0" >> ~/.bashrc && source ~/.bashrc
This command adds the display forwarding address to
.bashrc and makes the changes happen in the current shell
Additionally, there are some quality of life aliases that can be added to
> echo "alias ll='ls -lh --color=auto'" >> ~/.bashrc> # Long form list of directory contents, colorized, and human-readable
> echo "alias ls='ls -h --color=auto'" >> ~/.bashrc> # Human-readable and colorized listing of directory contents
> echo "alias grep='grep --color=auto'" >> ~/.bashrc> # Colorizes
> grep> results
> echo "alias update='sudo apt-get update && sudo apt-get upgrade'" >> ~/.bashrc> # Adds a command that will update and upgrade all packages at the same time. This is good to do from time to time.
> source ~/.bashrc> # starts the changes to
> .bashrc> in current terminal
Step 7: Links
Linux has full access to your Windows file system. However, it can be a little disorienting to find your way around. This is because Windows users are used to starting at the Desktop. To see your drives in the Ubuntu shell, they are found at:
Due to how illogical the system can be, here are a couple of commands to copy & paste into the Ubuntu shell that would make things so much easier.
ln -s /mnt/<drive letter> ~/<drive letter> # replace the < ... > with the drive letter name. For example:
ln -s /mnt/c ~/c
Depending on what your drive letters are, or how many you have, this command will make a 'shortcut' to those drives in your Ubuntu home directory. With this, we can start navigating through our Windows files like this:
Just as Linux has full access to the Windows file system, Windows has full access to Linux. However, where the Ubuntu home directory is stored (relative to the Windows file system) is somewhat confusing. For the sake of brevity, I will give you the path (if you installed Ubuntu through the store).
If you are root:
<username> is your Windows username.
If you are not root:
<username> is your Windows user name, and
<linux_username> is the Linux username that you setup when you first opened Ubunut.
For example, mine would look like:
I suggest you
e to open up a File Explorer, and paste:
C:\Users\<username>\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\ (one folder above your home directory) into the address bar and right click the name of your home directory, and select 'Pin to Quick access' (seen below).
Step 8: Restart and ready to roll
My next blog will be covering
conda env from the viewpoint of a bioinformaticist. This will introduce the ideas of why virtual environments are helpful, why you should use them, and basic project structures.
I hope you found this blog helpful. Feel free to post a comment below regarding critiques/criticisms, edits, and/or thanks. Look forward to hearing from you.
Additionally, I have posted examples of the config files that I use personally at https://github.com/betteridiot/blog. Please use & abuse them as you see fit. If you want to use them as is, just copy them into your home directory (in your linux terminal).