UPDATED: Windows 10 Subsystem for Linux (WSL) for Python developers

front_logo.png

Revisions

It has been over a year since my last blog post on how to setup the Windows Subsystem for Linux (WSL) for Python developers and I realized that I needed to update some of the steps.

The reason for the update is that WSL was still somewhat in its infancy when I first started exploring it, and Microsoft has taken tremendous strides towards making WSL a better feature. Additionally, I found refinements for a lot of steps in the "day 1" computer setup I use in my classes. Therefore, I would like to take the opportunity to share those developments here.


Tabula rasa redux

One of the hardest days in teaching introduction to bioinformatics material is the first day: Setting up your machine. This is difficult for one very simple reason, everyone tends to have a different one. While I have seen a very large bias towards Macs in academia, there are plenty of people that keep their Windows machines as a badge of pride...me included.

As a Python developer in an academic environment, I am a huge proponent of Anaconda builds. Anaconda has taken very large strides in making a seemingly ubiquitous environment for all platforms. However, because of the perceived enrichment of Macs in the academic field and most cloud computing being Unix-like systems, it is difficult to easily translate the platform differences at a finer grain level.

To ease the headaches of those users, this updated blog piece is the steps I take to set up my Windows 10 system for bioinformatics development.


The Setup

  • Windows 10 : version 1709 or later
  • Internet connection

Step 1: The Windows Subsystem for Linux

I am going to be suggesting a lot of keyboard shortcuts during this tutorial, so I figured I would list them here.


Win: Windows key
Search: Win + s
Run: Win + r

  • The first thing we need to do is make sure your Windows is up to date. Run winver and make sure the version is greater than or equal to 1709

winver

A new window should pop-up that looks like this:

winver_output

A Note about your Windows Version

If your Windows version is ≥ 1709, you can keep going with the rest of this blog.

If it is < 1709, I strongly recommend updating your computer.

Search for 'Turn Windows features'

feature search

  • ☑ Windows Subsystem for Linux

wsl check

  • Restart your machine (Win + x, then u + u)

Step 2: Installing Ubuntu

  • Search for 'Microsoft Store'
  • Look for 'Ubuntu' (or whatever Linux build you are interested in) when the store opens and click on the first option that shows up

win store

  • Press 'Get'

store get

  • Follow all prompts

For ease of use

I will refer to whatever Linux distribution you installed as Ubuntu from here on out, mainly to save keystrokes and because it is what I urge my students to install.


Quality of Life Stuff

As a developer, I am regularly opening new terminals. Therefore, I don't want to search in the Start menu for it every time. Thankfully, Windows 10 has a very strong keyboard shortcut system. To leverage it, do the following:

  1. Search for Ubuntu
  2. Right click on the result and select 'Pin to Taskbar' (shown below)
    pin
  3. Then move the pin to the far left, just next to the Windows Logo
    taskbar

With this in place, the user can now use Win + 1 to open a terminal. All Win + 1 does is open the first program pinned to the taskbar.


Step 3: Setting up Ubuntu

Search for Ubuntu, and run it once. It will say 'Installing, this may take a few minutes...'. This step needs to be done for the whole file system to be setup.

User Accounts

Once the setup step is complete, Ubuntu will asks you for a Username and password, and you have two choices:

  1. close it (if you want to be root by default)
  2. fill it out (if you don't want to be root by default)

Whichever you choose is up to you, and the only real things this affects (since this is your own computer and not a shared system) is what your $HOME directory will be named and whether or not you have to type su[do] for some commands.

I, personally, use the first method (I think I just gave a sysadmin an aneurysm somewhere).


Quality of Life Stuff

File Convenience

If you haven't done a lot of stuff on your computer, you may need to change some settings to make things a little easier. To make hidden files visible and to show file extensions, do the following:

  • Win + e to open a file explorer
  • Click on the View tab
  • check both 'File name extensions' and 'Hidden items'

extensions

As it stands, Ubuntu has full access to the Windows file system, and Windows has semi-full access to Ubuntu file system.
Now we just need to figure out how to find the Ubuntu files on Windows, and it is kind of confusing at the moment (a future release of Windows will be fixing this soon).

For the sake of brevity, I will give you the path (if you installed Ubuntu through the store).

User accounts

This is one of the few places in the walk through that is affected by whether or not you chose to be root

If you are root:

C:\Users\%USERNAME%\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\root

If you are not root:

C:\Users\%USERNAME%\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\home\UBUNTU_USERNAME

Where UBUNTU_USERNAME is the Ubuntu username that you setup when you first opened Ubuntu.

My suggestion

Use Win + e to open up a File Explorer, and paste: C:\Users\%USERNAME%\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\ (one folder above your Ubuntu home directory) into the address bar.

If you are root, right click the root folder here, and "Pin to Quick Access" (Shown below).

If you are not root, go into the home folder, and then right-click your Ubuntu user name folder, and "Pin to Quick Access"


BIG CAVEAT

One of the biggest issues with the current WSL implementation is how Windows "shares" with Ubuntu. Ubuntu can view, modify, and create files/folders on both Ubuntu and Windows alike. However, Windows can only view and modify files/folders in the Ubuntu file system (that folder we just pinned to Quick Access). It cannot create files in the Ubuntu file system. Again, this is fixed in a future release of Windows, there is a good rule to remember:

If Ubuntu builds it, Windows will come.

Generally speaking, it is just good to have shortcuts within Ubuntu that direct you somewhere other than its file system (that folder we pinned to Quick Access earlier). That way it doesn't matter where the file originates from.


Step 4: Making Ubuntu's $HOME feel like home

We want to make Ubuntu have some good creature comforts and allow it act responsively with Windows programs. To do this, I have created a helpful GitHub repository chock-full of useful settings that you don't have to agonize over.

Below are the steps to follow that will allow your computer to work seamlessly with Jupyter Notebooks/Lab and even leverage the power of Microsoft Office programs on files created within your Ubuntu system.

Step 4.1: Installing Miniconda

Miniconda is a minimal build of Anaconda. While I am a huge proponent of Anaconda, it is often a little too bulky to be used as a base environment. To get & install the latest version of Miniconda, open Ubuntu (from here on, called the "terminal") and copy & paste these steps into the terminal:

# Make a place to store our downloaded files
mkdir Downloads && cd Downloads

# Download and install Miniconda
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

This starts the install process of Miniconda. There are some prompts to follow. Most of it is pretty obvious, but for verbosity's sake:

  1. Press Enter (accept default) when asked to install
  2. Press the spacebar to proceed through the license until it assk you something else
  3. Type yes when asked to accept the license
  4. Press Enter (accept default) when it asks you where to install Miniconda
  5. If it asks you if you want to install all the packages above, type yes
  6. When it asks "Do you wish the installer to initialize Minicond by running conda init? " type yes

With that done, we can do a quick check to see if it all worked:

cd && source ~/.bashrc

Miniconda install check

If, after you typed that last command, your prompt (the place you type commands) changed to include (base) in the front, you have successfully installed Miniconda!

Step 4.2: Setting up your shortcuts & aliases

At times, it can be difficult to find your way around the Windows/Ubuntu file system. Furthermore, there are some simple aliases (shortcuts for regularly used commands) that are super helpful in navigation. So, we are going to set them all up.

First, though, I want to explain what we will be adding. This is for 2 reasons: 1) so you, the user, knows what will be available to you, and 2) for transparency of what we will be changing.

  1. Making shortcuts to your Windows file system
    • This command will locate all of your drives (C:, D:, etc) and add shortcuts to them in your $HOME directory

      for dir in /mnt/*; do ln -s $dir $(basename $dir); done

  2. Update conda and setup your base environment to have Jupyter ready to go

    • These commands will initialize your system to play nice with Jupyter Lab/Notebook since they are browser-based applications and Ubuntu doesn't have a browser.

      • The first will download a jupyter config file that allows you to use Jupyter as root and increases the IOPub rate.

        mkdir ~/.jupyter && cd ~/.jupyter && wget https://raw.githubusercontent.com/betteridiot/terminal_support/master/.jupyter/jupyter_notebook_config.py

      • The second downloads a conda config file that will allow the user to install packages from third-party libraries (conda-forge and bioconda)

        cd && wget https://raw.githubusercontent.com/betteridiot/terminal_support/master/.condarc

      • The third command will add the following aliases to your .bashrc (a file that just tell the terminal how to initialize every time it opens):
        1. ls: A colorized and sorted version of ls
        2. grep: A colorized version of grep
        3. ll: A colorized, sorted, human-readable, long form list version of ls
        4. update: A shortcut to update your Ubuntu distribution
        5. jlab: A simple shortcut that opens up JupyterLab in your $HOME directory in the background
        6. jnote: A simple shortcut that opens up Jupyter Notebook in your $HOME directory in the background

          curl https://raw.githubusercontent.com/betteridiot/terminal_support/master/helpful_aliases >> ~/.bashrc && source ~/.bashrc

      • Finally, let's enable our jupyter development environment. The first command updates conda, and the second installs the very specific environment build that we need to use jupyter from our Ubuntu system on our Windows Browser

        conda update conda

        conda install "python>=3.7" numpy pandas matplotlib seaborn jupyter jupyterlab notebook=5.7.2 "tornado<6"

4.3: Linking your browser to Ubuntu

The last part of the equation is how to use your Windows-side internet browser with your Ubuntu-side Jupyter. I will be using Google Chrome for the browser to be used. However, you can use just about any browser, so long as you follow these steps.

  1. Search for 'Chrome', right-click it and select 'Open file location'

    chrome

  2. In the window that pops up, Google Chrome should already be highlighted. Right click it and select 'Properties'

    properties

  3. In the window that pops up, copy the text under the 'Target' field

    target

  4. Replace GOOGLE_CHROME of the following command with the text copied from Google Chrome, and paste the command into the terminal:

    echo export BROWSER=\"$(wslpath GOOGLE_CHROME)\" >> ~/.bashrc && source ~/.bashrc


Conclusion

I know it seems like a lot of steps, and it kind of is. However, they were mostly pretty simple. The postive note though is that you should now have full control over your WSL-based Python environment.

Testing Jupyter

To test your shiny new Jupyter environment on your brand new Ubuntu system, type this custom-built alias into your terminal

jlab

After a couple of seconds, your computer should open up the browser you selected to use and start up JupyterLab for your programming enjoyment.

Thanks for taking the time to go through this with me. If you feel like this was a helpful guide, feel free to share it using any of the social media buttons below. Also, consider subscribing to my blog to receive notifcations when new content is posted.

Have fun and code responsibly!

Name *
Name