Using WSL for AI App Development

Linux is an open-source, Unix-like operating system kernel created by Linus Torvalds in 1991. It was inspired by Unix but developed as a free and open alternative. Today, Linux powers servers, cloud infrastructure, Android devices, IoT, cybersecurity, DevOps, and AI research.

Recently we have started working on a research project with a high level requirements as under:
✅ Crawls public websites for topics of your interest ( e.g. Java ).
✅ Extracts potential Java questions and generates high-quality answers consulting multiple LLMs.
✅ Uses LLMs (Large Language Models) for Q&A generation, topic classification, and logical error analysis.
✅ Runs a scheduled batch inference processing system for efficiency.
✅ Uses PostgreSQL for structured data storage and LLMs for semantic search.
✅ Optimizes inference performance on a local Linux environment using WSL and SSD-based swap.
✅ Implements benchmarking & auto-routing for cost-effective LLM selection (local hosted models and cloud hosted like Groq, GPT-4).

Virtual Box vs WSL as our Linux platform

Linux is the no brainer choice to develop , test and deploy but many developers, especially freshers and juniors, are more familiar with Windows and may not be comfortable setting up a dual-boot partition or getting a separate Linux machine. Given these constraints we had 2 choices either go with WSL or go with VirtualBox. Here is a brief comparison of WSL vs VirtualBox

Performance

WSL is faster for lightweight tasks because it shares Windows resources.
VirtualBox is slower since it runs a full virtual machine, requiring more system overhead.

Resource Usage

WSL uses minimal resources and runs as a lightweight layer on Windows.
VirtualBox allocates dedicated CPU, RAM, and storage, leading to higher resource consumption.

Kernel & System Architecture

WSL 2 includes a built-in Linux kernel managed by Windows.
VirtualBox runs a full independent Linux OS inside a VM.

Networking

WSL 1 shares Windows’ network stack, while WSL 2 has its own virtual network interface.
VirtualBox runs on a separate virtual network adapter, allowing full network isolation.

Use Case

WSL is ideal for development, scripting, AI/ML workloads, and DevOps tools without the overhead of a VM.
VirtualBox is better for running full Linux distributions with GUI applications and system-level testing.

Hardware Access

WSL has limited access to hardware, though WSL 2 offers GPU acceleration.
VirtualBox provides full access to USB, GPU (with proper drivers), and other hardware devices.

GUI Support

WSL now includes WSLg, which allows running Linux GUI apps natively.
VirtualBox supports full Linux desktop environments, making it better for GUI-based applications.

Installation & Setup

WSL is quick to set up via Windows features.
VirtualBox requires ISO downloads, manual installation, and configuration. We decided to go with WSL2 is very quick and easy to setup and WSL2 is quite matured and stable. In the rest of the article we will cover WSL1 vs WSL2, how to install and do some advanced configurations

WSL 1 vs. WSL 2: Key Differences

Kernel:

WSL 1 uses the Windows NT kernel.
WSL 2 runs a full Linux kernel inside a lightweight virtual machine.

Performance:

WSL 1 is faster for file system operations.
WSL 2 performs better for system calls and running Linux applications.

Resource Usage:

WSL 1 uses fewer system resources.
WSL 2 requires more resources but provides better functionality.Networking: WSL 1 shares Windows’ network stack.
WSL 2 has a separate network interface, like a virtual machine.

Best For:

WSL 1 is best for simple command-line tools.
WSL 2 is better for running full Linux applications and services.
We shortlisted WSL 2 because we will be working Docker, AI workloads, databases, and system-level Linux tools.

Installing and Setting Up WSL on Windows

Check If WSL Is Installed

Run the following command in PowerShell:


                    wsl --list --verbose

📌 If WSL is installed, you’ll see a list of distributions.
📌 If not installed, install it with:
wsl --install
Default Linux distribution installed: Ubuntu

Verify Installation

From command line run:
wsl --version
Expected Output:
WSL version: 2.4.10.0 Kernel version: 5.15.167.4-1

For our team, it is important that we keep coding, browsing, emailing and attending meetings on Windows while WSL in the background
— Crawls the web looking for open source content on our topic of interest ( example java ).
— Runs the LLMs ( ollama / Mistral 7B model ) extract questions, answers , summarizing the content.
— Find duplicates and discard them using pgVector database.
— insert the responses from LLM into postgres database also running in WSL.

Challenge is running local LLMs on CPU ( no GPUs ) is both CPU and Memory intensive. Inference especially is quite memory intensive and we do want to have as large a model as possible. By default, WSL can consume a large portion of system memory, impacting Windows performance where we do our meetings, write emails and code. So it is necessary we do following

Set an upper memory limit on how much RAM and CPU WSL can consume ( We followed the 50:50 rule giving 50% for WSL , 50% for Windows ).

Running local AI models (e.g., llama.cpp or mistral-7b) is memory-intensive and not practical even for our use case where LLM inferences happen in background jobs unless we get more substantially more RAM to WSL.

So how do you achieve the above ? By modify the WSL global settings file (.wslconfig) to manually limit and optimize resource allocation & allocate more SWAP from your SSD or HDD : [wsl2] memory=8GB # Limit RAM usage to 50% (default is dynamic) processors=4 # Restrict to 50% CPU cores swap=45GB # Enable swap on SSD (for heavy workloads) swapFile=C:\wsl-swap.vhdx # Swap file location

Apply the changes by restarting WSL:
wsl --shutdown

Testing Resource Limits with stress Command

To verify if WSL is correctly applying CPU, RAM, and swap settings, use the stress tool to simulate high workloads.
Installing stress in WSL
sudo apt update && sudo apt install -y stress
To stress-test CPU usage, run:
stress --cpu 4 --timeout 60
This should only utilize 4 cores (as set in .wslconfig) and exit after 60 seconds.
To stress test RAM usage allocate 7GB of RAM (just below the 8GB limit):
stress --vm 1 --vm-bytes 7G --timeout 60
This ensures WSL stays within the allocated memory.
Testing Swap File Usage
To force memory pressure and test swap activation:
stress --vm 1 --vm-bytes 10G --timeout 60
If swap is properly configured, the system should not crash and should start using SSD-based swap.

Conclusion

WSL is a game-changer for developers needing Linux tools while working on Windows. By configuring resources efficiently, optimizing disk usage, you can unlock the full power of WSL & even use to develop and run AI Apps on your regular Windows Laptop.