Java Concurrency 101: Understanding Multithreading Fundamentals

Imagine you’re using your favorite code editor on a computer with a single processor. When you make changes to a code file and click “save,” a series of actions is triggered, ultimately writing data to the computer’s hard drive.

Now, here’s the catch: this writing process, known as I/O (Input/Output), can be quite slow, and during this time, the computer’s CPU (the brain of the computer) is basically waiting and not doing anything productive.

Instead of letting the CPU sit idle while waiting for I/O to finish, we can use threads. In this case, one thread is responsible for handling I/O, and another thread takes care of the user interface (UI).

So, when you click somewhere else on the screen while the I/O thread is busy, the CPU can switch to the UI thread, ensuring that your code editor remains responsive and doesn’t appear frozen or unresponsive to your actions. This way, your computer can multitask efficiently, even with a single processor.

Program, Process & Thread

Program:

• Programs can be things like applications, software, or scripts.
• To actually use a program, your computer’s operating system takes these instructions and data and creates something called a “process.”

Imagine you have a video game installed on your computer. The video game itself is like a program.

Process:

• Each process has its own dedicated resources like memory, CPU time, and storage.
• Multiple processes can run at the same time, each doing its own thing, and they don’t usually share resources with each other

Now, when you decide to play that video game, your computer’s operating system creates a process specifically for running the game.

Thread:

• A process can have multiple threads, and they work together to get things done. Threads within a process can share some information, but they also have their private data in stack.
• When threads work together, they need to be careful about sharing and accessing data to avoid problems.

Inside the game’s process, there are different tasks happening simultaneously. For example, one thread could be responsible for rendering graphics, another for handling player input, and yet another for managing the game’s sound. These threads work together within the game’s process.

Concurrency vs Parallelism

A concurrent program can be broken down into individual components, and each of these components can be executed independently or in a partially ordered manner without impacting the final result.

A classic example of a concurrent system is an operating system running on a single-core machine. In this scenario, the operating system is concurrent but not parallel. It can handle multiple tasks, but at any given moment, it can only process one task. Each task is allocated a portion of CPU time to execute and advance.

A parallel system is designed to execute multiple programs simultaneously. Typically, this capability is facilitated by hardware, such as multicore processors in individual machines or computing clusters where multiple machines collaborate to solve independent parts of a problem simultaneously.

Synchronous vs Asynchronous

Synchronous Execution in Java: In synchronous Java code, operations are performed one after the other. This means that if an operation takes a long time to execute, it will block the entire program, making it unresponsive. For example, consider a scenario where you need to download a file from the internet synchronously:

public void downloadFile() {
    // Blocking operation to download a file
    File file = downloadFromInternet();
    
    // Further operations with the downloaded file
    processFile(file);
}

In this case, the downloadFile method will not return until the file download is complete, potentially causing the application to appear frozen during the download.

Asynchronous Execution in Java: Asynchronous programming in Java allows you to perform operations concurrently, without waiting for one operation to complete before starting the next.

Java provides mechanisms like threads and the CompletableFuture framework to achieve asynchronous execution.

import java.util.concurrent.CompletableFuture;

public void downloadFileAsynchronously() {
    CompletableFuture.supplyAsync(() -> {
        // Asynchronous operation to download a file
        File file = downloadFromInternet();
        return file;
    }).thenAcceptAsync((file) -> {
        // Further operations with the downloaded file
        processFile(file);
    });
}

In this asynchronous example, we use CompletableFuture to initiate the download operation in the background. While the file is being downloaded, the program can continue executing other tasks. Once the download is complete, the thenAcceptAsync callback handles further processing of the file.

Asynchronous programming is an excellent choice for applications that do extensive network or disk I/O and spend most of their time waiting. As an example, Javascript enables concurrency using AJAX library’s asynchronous method calls.

CPU Bound & I/O Bound Tasks:

CPU-Bound Programs: CPU-bound programs are those that heavily rely on the CPU’s processing power, often utilizing it close to 100%. These programs perform computations that require significant CPU resources. Here are some examples:

Image Processing: Tasks like image filtering, resizing, or rendering that involve extensive pixel-level calculations.

Matrix Multiplication: Operations involving large matrices where each element requires computation.

Improving Performance:

• Implement multi-threading or parallel processing to utilize multiple CPU cores efficiently.
• Employ hardware acceleration or specialized libraries for specific tasks, like GPU computing for certain calculations.

public class CPUBoundExample {
    public static void main(String[] args) {
        int result = performIntensiveCalculations();
        System.out.println("Result: " + result);
    }

    public static int performIntensiveCalculations() {
        int sum = 0;
        for (int i = 1; i <= 1000000; i++) {
            sum += i;
        }
        return sum;
    }
}

I/O-Bound Programs: I/O-bound programs, on the other hand, spend a significant portion of their time waiting for input/output (I/O) operations to complete. During this waiting time, the CPU remains relatively idle. Examples of I/O operations include reading from or writing to files, making network requests, or interacting with a database.

Improving Performance:

• Use asynchronous programming models, such as Java’s CompletableFuture or reactive programming, to allow tasks to take control to the CPU while waiting for I/O, enabling concurrency.
• Employ non-blocking I/O operations, like Java NIO (New I/O), to minimize thread blocking during I/O operations.
• Implement caching mechanisms to reduce the frequency of I/O operations by storing frequently accessed data in memory.

public class IOBoundExample {
    public static void main(String[] args) throws IOException {
        long startTime = System.currentTimeMillis();
        String webpageContent = fetchWebpage("https://example.com");
        long endTime = System.currentTimeMillis();

        System.out.println("Webpage length: " + webpageContent.length());
        System.out.println("Time taken: " + (endTime - startTime) + " milliseconds");
    }

    public static String fetchWebpage(String url) throws IOException {
        URL webpageUrl = new URL(url);
        StringBuilder content = new StringBuilder();

        try (BufferedReader reader = new BufferedReader(new InputStreamReader(webpageUrl.openStream()))) {
            String line;
            while ((line = reader.readLine()) != null) {
                content.append(line);
            }
        }
        return content.toString();
    }
}

In this Java example, the fetchWebpage method performs an I/O-bound task by making an HTTP request to a website. During the network request, the CPU can be utilized for other tasks, as it doesn't need to wait for the I/O operation to complete.

Throughput vs. Latency:

In the context of a web server, Throughput is a measure of how many requests can be processed per second. For example, if a web server can handle 1000 requests per second, its throughput is 1000 requests/second.

Optimizing throughput often involves parallelizing tasks and efficiently using available resources.

Latency, on the other hand, is the time it takes for a single request to be processed and for a response to be sent back to the client. It’s a measure of the responsiveness of the server. If a web server has a latency of 50 milliseconds for a specific request, it means that it takes 50 milliseconds to complete that request and send a response.

To reduce latency, techniques like caching, content delivery networks (CDNs), and minimizing round-trip network requests can be employed. Additionally, optimizing database queries and using faster hardware can help reduce latency.

Critical Sections & Race Conditions:

A critical section is a portion of a program or code that can be executed concurrently by multiple threads within an application.

Critical sections are used to protect shared resources in a multi-threaded environment. They ensure that only one thread can access the critical section at a time, preventing concurrent modifications.

Prevention: Techniques like locks, semaphores, or mutexes are used to implement critical sections. Proper synchronization is crucial to avoid conflicts.

A race condition occurs when multiple threads access shared resources or program variables simultaneously,

Prevention: To mitigate race conditions, synchronization mechanisms such as locks or atomic operations are employed.

Race Condition Scenario: Consider a banking application where multiple threads are responsible for transferring money between different bank accounts. This scenario involves shared resources, such as the bank account balances, which must be protected to ensure data consistency and avoid race conditions.

public class RaceConditionExample {
    public static void main(String[] args) {
        BankAccount account = new BankAccount(1000);

        Runnable depositTask = () -> {
            for (int i = 0; i < 100; i++) {
                account.deposit(10);
            }
        };

        Runnable withdrawTask = () -> {
            for (int i = 0; i < 100; i++) {
                account.withdraw(10);
            }
        };

        Thread depositThread1 = new Thread(depositTask);
        Thread depositThread2 = new Thread(depositTask);
        Thread withdrawThread1 = new Thread(withdrawTask);
        Thread withdrawThread2 = new Thread(withdrawTask);

        depositThread1.start();
        depositThread2.start();
        withdrawThread1.start();
        withdrawThread2.start();
    }
}

Here multiple threads deposit and withdraw funds from the same account without synchronization. This can result in race conditions where the balance is not updated correctly, leading to incorrect account balances.

Race Condition Solution (Using synchronized/MUTEX): In this code, the deposit and withdraw methods are synchronized, ensuring that only one thread can execute them at any given time. This synchronization prevents race conditions when multiple threads attempt to modify the account balance concurrently.

class BankAccount {
    private int balance;

    public BankAccount(int initialBalance) {
        this.balance = initialBalance;
    }

    public synchronized void deposit(int amount) {
        // Critical section: Ensures synchronized access to the balance
        balance += amount;
        System.out.println("Deposited " + amount + " dollars. New balance: " + balance);
    }

    public synchronized void withdraw(int amount) {
        // Critical section: Ensures synchronized access to the balance
        if (balance >= amount) {
            balance -= amount;
            System.out.println("Withdrawn " + amount + " dollars. New balance: " + balance);
        } else {
            System.out.println("Insufficient funds.");
        }
    }
}

Race Condition Solution (Using ReentrantLock/MUTEX):

import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;

class BankAccount {
    private int balance;
    private Lock lock;

    public BankAccount(int initialBalance) {
        this.balance = initialBalance;
        this.lock = new ReentrantLock();
    }

    public void deposit(int amount) {
        lock.lock(); // Acquire the lock
        try {
            balance += amount;
            System.out.println("Deposited " + amount + " dollars. New balance: " + balance);
        } finally {
            lock.unlock(); // Release the lock in a finally block
        }
    }

    public void withdraw(int amount) {
        lock.lock(); // Acquire the lock
        try {
            if (balance >= amount) {
                balance -= amount;
                System.out.println("Withdrawn " + amount + " dollars. New balance: " + balance);
            } else {
                System.out.println("Insufficient funds.");
            }
        } finally {
            lock.unlock(); // Release the lock in a finally block
        }
    }
}

Thread Safety in Java

A class and its public APIs are labelled as thread safe if multiple threads can consume the exposed APIs without causing race conditions or state corruption for the class.

Thread-unsafe conditions occur in a multi-threaded program when multiple threads access shared resources or data concurrently, and their operations on that data can lead to unpredictable and incorrect behavior.

Scenarios of Unsafe Thread & Solutions:

Race Conditions: A race condition occurs when two or more threads access shared data and try to modify it concurrently.

// Thread 1
counter++;

// Thread 2
counter++;

Solution: Synchronized keyword

// Shared counter
private static int counter = 0;

// Synchronized method to increment counter
public synchronized void incrementCounter() {
    counter++;
}

Deadlocks: A deadlock is a situation where two or more threads are unable to proceed because they are waiting for each other to release a resource. This can result in a complete program freeze.

// Thread 1
synchronized (lock1) {
    // Do something with lock1
    synchronized (lock2) {
        // Do something with lock2
    }
}

// Thread 2
synchronized (lock2) {
    // Do something with lock2
    synchronized (lock1) {
        // Do something with lock1
    }
}

Solution: Modify your code so that all threads acquire locks in the same order according to the global order you defined

// Thread 1
synchronized (lock1) {
    // Do something with lock1
    synchronized (lock2) {
        // Do something with lock2
    }
}

// Thread 2
synchronized (lock1) {
    // Do something with lock1
    synchronized (lock2) {
        // Do something with lock2
    }
}

Resource Contention: This problem typically arises in multi-user database environments where concurrent access to data is required.

Scenario: In a database system, various resources are shared among multiple users or threads, including:

Database Connection: Limited database connections can be a common resource that multiple client applications or threads compete for.

Locks: Locks are used to control access to data rows or tables. When multiple transactions try to acquire locks on the same data simultaneously, it can lead to contention.

Buffer Pool: The buffer pool is used to cache frequently accessed data pages.

Solution: Use connection pooling mechanisms to manage and reuse database connections efficiently. This reduces the overhead of establishing a new connection for each request.

For extremely high-traffic applications consider database sharding, distributed database environment consider load balancing

// Using a connection pool (example with HikariCP)
HikariConfig config = new HikariConfig();
config.setJdbcUrl("jdbc:mysql://localhost/testdb");
config.setUsername("username");
config.setPassword("password");

HikariDataSource dataSource = new HikariDataSource(config);

// Use dataSource to get database connections
Connection connection = dataSource.getConnection();

Volatile keyword in Java:

Visibility Problem Without Volatile: Consider a scenario where multiple threads are accessing a shared variable without using volatile. Each thread may cache the variable's value locally (in the thread’s local cache) which is only private to specific thread, and updates made by one thread might not be immediately visible to other threads. This can lead to inconsistent or outdated data being read by different threads.

Using Volatile for Visibility: By declaring a variable as volatile, you indicate that its value should always be read and written directly from and to the main memory (RAM), which is accessible by all thread. This ensures that changes to the variable made by one thread are immediately visible to all other threads.

• One thread updates volatile, other threads can see it. volatile is well-suited for scenarios where variables are read frequently but modified infrequently. For example, flags or status indicators that control the flow of a program can benefit from volatile.

public class VolatileFlagExample {
    private volatile boolean flag = false;

    public void start() {
        // Thread 1: Reader thread
        Thread readerThread = new Thread(() -> {
            while (!flag) {
                // Wait until the flag becomes true
            }
            System.out.println("Reader Thread: Flag is now true.");
        });

        // Thread 2: Writer thread
        Thread writerThread = new Thread(() -> {
            try {
                // Simulate some work
                Thread.sleep(2000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            
            // Set the flag to true when the condition is met
            flag = true;
            System.out.println("Writer Thread: Flag is set to true.");
        });

        // Start both threads
        readerThread.start();
        writerThread.start();
    }

    public static void main(String[] args) {
        VolatileFlagExample example = new VolatileFlagExample();
        example.start();
    }
}

Without the volatile keyword, the reader thread might cache the value of the flag variable, leading to a situation where it never detects the change made by the writer thread.

Volatile vs Synchronized : Use Case Differences

Use volatile when you have a variable that is read frequently but modified infrequently, and you want to ensure that changes to this variable are immediately visible to other threads. A common use case is for flags or status indicators.

Use synchronized when you need to protect critical sections of code to achieve mutual exclusion among threads. It ensures that only one thread can execute the synchronized block at a time, making it suitable for scenarios where multiple threads may concurrently access shared resources.

While volatile ensures visibility, it doesn't provide atomicity. If multiple threads both read and write to a volatile variable, additional synchronization mechanisms like synchronized or java.util.concurrent classes should be used to ensure both visibility and atomicity.

Modern Concurrency Frameworks: CompletableFuture

Modern Concurrency Frameworks like CompletableFuture and libraries like RxJava provide powerful tools for managing concurrency, handling asynchronous operations, and composing complex asynchronous workflows.

CompletableFuture (Java 8+):

It’s useful when you want to perform multiple asynchronous operations and handle their results or errors efficiently. It allows you to create complex asynchronous workflows.

Imagine a scenario where you need to fetch data from multiple web services concurrently and aggregate the results.

import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;

public class CompletableFutureExample {
    public static void main(String[] args) throws InterruptedException, ExecutionException {
        CompletableFuture<String> future1 = CompletableFuture.supplyAsync(() -> fetchDataFromService("Service1"));
        CompletableFuture<String> future2 = CompletableFuture.supplyAsync(() -> fetchDataFromService("Service2"));

        CompletableFuture<Void> combinedFuture = CompletableFuture.allOf(future1, future2);

        combinedFuture.thenRun(() -> {
            try {
                String result1 = future1.get();
                String result2 = future2.get();
                System.out.println("Combined Result: " + result1 + " and " + result2);
            } catch (InterruptedException | ExecutionException e) {
                e.printStackTrace();
            }
        });

        // Prevent the main thread from exiting before completing the futures
        Thread.sleep(2000);
    }

    private static String fetchDataFromService(String serviceName) {
        // Simulate fetching data from a web service
        return serviceName + " data";
    }
}

ConcurrentHashMap:

When multiple threads need to read and write key-value pairs concurrently without introducing synchronization overhead or risking data corruption.

• Solution: ConcurrentHashMap internally divides the map into segments, allowing multiple threads to operate on different segments simultaneously. This minimizes contention and locks only the relevant segments during write operations, ensuring thread-safety without blocking.

Use Case: Concurrently storing user sessions in a web application where multiple threads need to read and update sessions without locking. ConcurrentHashMap can be used as a cache to store frequently accessed data.

// Create a ConcurrentHashMap to store user sessions
ConcurrentHashMap<String, UserSession> sessionMap = new ConcurrentHashMap<>();

// User logs in, and their session is added or updated
String sessionId = "user123";
UserSession userSession = new UserSession(sessionId);
sessionMap.put(sessionId, userSession);

// Later, a different thread retrieves the session
UserSession retrievedSession = sessionMap.get(sessionId);

ConcurrentLinkedQueue:

• Use Case: When you need a queue for managing tasks or data that must be processed concurrently by multiple threads without causing contention. It’s commonly used in thread pool implementations.
• Example: A task queue in a thread pool where multiple worker threads pull and process tasks concurrently.

// Create a ConcurrentLinkedQueue for managing tasks
ConcurrentLinkedQueue<Runnable> taskQueue = new ConcurrentLinkedQueue<>();

// Multiple producer threads enqueue tasks
taskQueue.offer(() -> { /* Task 1 */ });

// Multiple consumer threads dequeue and execute tasks
Runnable task = taskQueue.poll();
task.run();