1. Introduction:

Linux applications frequently rely on threads to handle multiple tasks in parallel. A system may process data from hardware interfaces while handling network traffic and performing background work at the same time. Threads are introduced to keep the application responsive and to prevent slow operations from blocking the main execution path. Once threads are added, however, memory is often shared between execution contexts.

Shared memory introduces a class of problems that depend on timing rather than logic. A program may behave correctly during testing and fail only under specific load conditions or after running for long periods of time. Small changes such as adding debug output or running on different hardware can alter the execution order enough to hide the problem entirely. This makes concurrency bugs difficult to reproduce and even harder to debug in embedded environments.

ThreadSanitizer provides a practical way to detect these issues by monitoring how threads access shared memory at runtime. Instead of relying on manual inspection of locking logic, it reports conflicting memory accesses while the program is running, making data races visible during developmen

2. Data Races in Linux Applications :

A data race occurs when multiple threads access the same memory location at the same time and at least one of those accesses modifies the data without proper synchronization. In user-space Linux programs this usually involves global variables, shared buffers, or state objects that are visible to more than one thread. When no clear ordering exists, the final behavior depends on how the kernel schedules threads and how the processor reorders memory operations.

This situation commonly appears in producer–consumer designs. One thread collects data from a sensor or device driver, while another thread processes that data asynchronously. Coordination is often implemented using a shared flag that indicates when new data is available. The following pseudocode illustrates this pattern:

shared ready_flag = false
shared data_buffer

thread Producer:
    loop forever:
        data_buffer = read_sensor_or_fill_buffer()
        ready_flag = true

thread Consumer:
    loop forever:
        if ready_flag == true:
            local_copy = data_buffer
            ready_flag = false
            process(local_copy)

Although this logic appears straightforward, the absence of synchronization means that memory updates can be observed out of order or concurrently. The producer may set the flag before the buffer contents are fully visible, or the consumer may read the buffer while it is still being modified. These errors depend on timing rather than logic and may not appear consistently during testing.

In larger and more complex projects, where shared state is accessed through multiple execution paths, such timing-dependent issues become extremely difficult to detect through code inspection or testing alone, which is why a runtime tool like ThreadSanitizer is needed.

3.ThreadSanitizer Overview and Internal Operation:

ThreadSanitizer works by instrumenting the program during compilation. When the application is built with ThreadSanitizer enabled, the compiler inserts additional code around every memory read, write, and synchronization operation. These extra instructions do not change the program logic, but they allow the runtime to observe how memory is accessed by different threads.

At runtime, ThreadSanitizer maintains a shadow memory that mirrors the program's address space. For each memory location, the shadow data records metadata describing which thread last accessed it and under what synchronization context. When a thread performs a read or write, the instrumentation updates this shadow state and checks whether another thread has accessed the same location without an ordering relationship defined by a lock or atomic operation.

If two threads access the same memory location and no proper synchronization is observed, ThreadSanitizer reports a data race. The report includes the memory address and the call stacks of the threads involved, making it possible to trace the problem back to the exact code paths that caused it. A diagram in this section illustrates how memory access history is recorded in shadow memory and checked at runtime.

This mechanism adds execution overhead and increases memory usage, which is why ThreadSanitizer is used only during development.

4.Using ThreadSanitizer in Linux:

ThreadSanitizer is enabled at build time and runs as part of the application during execution. In embedded Linux projects it is typically used in debug builds, where the additional memory usage and execution overhead are acceptable. A common scenario involves a producer–consumer design in which one thread fills a shared buffer and signals availability using a flag, while another thread waits for that flag and consumes the data. The following example illustrates this pattern using two POSIX threads and intentionally omits synchronization to demonstrate a data race.

#include
#include
#include
#include
#include

/* shared resources (intentionally unsynchronized) */
static int ready_flag = 0;
static uint8_t data_buffer[32];

static void* producer(void* arg)
{
    (void)arg;
    while (1) {
        /* fill buffer */
        for (uint32_t i = 0; i < sizeof(data_buffer); ++i) {
            data_buffer[i] = (uint8_t)i;
        }
        /* publish */
        ready_flag = 1;
        usleep(1000);
    }
    return NULL;
}

static void* consumer(void* arg)
{
    (void)arg;
    while (1) {
        if (ready_flag) {
            uint8_t local[32];
            memcpy(local, data_buffer, sizeof(local));
            ready_flag = 0;
            /* tiny use to avoid optimizing away */
            printf("got %u %u %u\n", local[0], local[1], local[2]);
        }
    }
    return NULL;
}

int main(void)
{
    pthread_t t1, t2;
    pthread_create(&t1, NULL, producer, NULL);
    pthread_create(&t2, NULL, consumer, NULL);
    pthread_join(t1, NULL);
    pthread_join(t2, NULL);
    return 0;
}

This is a simple build that works well for development. It keeps optimization low and includes debug symbols so the report contains usable stack traces.

CC ?= clang
CFLAGS ?= -O1 -g -fno-omit-frame-pointer
LDFLAGS ?=
LDLIBS ?= -pthread

# ThreadSanitizer flags
TSANFLAGS = -fsanitize=thread

all: race_demo

race_demo: race_demo.c
    $(CC) $(CFLAGS) $(TSANFLAGS) $< -o $@ $(LDLIBS) $(LDFLAGS)

clean:
    rm -f race_demo

When a race is detected, ThreadSanitizer prints a report similar to the following. The key part is that it shows the same address, the two threads, and where the accesses happened.

==================
WARNING: ThreadSanitizer: data race (pid=37772)
  Read of size 4 at 0x576f5f4a6060 by thread T2:
    #0 consumer /root/thread/race_demo.c:32 (race_demo+0x12fc) (BuildId: 5917d7de2e9cd7c811b20bae43eca035978413d3)

  Previous write of size 4 at 0x576f5f4a6060 by thread T1:
    #0 producer /root/thread/race_demo.c:20 (race_demo+0x1395) (BuildId: 5917d7de2e9cd7c811b20bae43eca035978413d3)

  Location is global 'ready_flag' of size 4 at 0x576f5f4a6060 (race_demo+0x4060)

  Thread T2 (tid=37775, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1022 (libtsan.so.2+0x5ac1a) (BuildId: 38097064631f7912bd33117a9c83d08b42e15571)
    #1 main /root/thread/race_demo.c:49 (race_demo+0x1430) (BuildId: 5917d7de2e9cd7c811b20bae43eca035978413d3)

  Thread T1 (tid=37774, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1022 (libtsan.so.2+0x5ac1a) (BuildId: 38097064631f7912bd33117a9c83d08b42e15571)
    #1 main /root/thread/race_demo.c:48 (race_demo+0x1413) (BuildId: 5917d7de2e9cd7c811b20bae43eca035978413d3)

SUMMARY: ThreadSanitizer: data race /root/thread/race_demo.c:32 in consumer
==================

5. Conclusion:

Threading is common in embedded Linux systems, but shared state makes correctness depend on timing and scheduling. Data races can remain hidden during testing and become almost impossible to track down as a project grows. ThreadSanitizer makes these problems visible by reporting unsynchronized memory accesses with clear stack traces, allowing races to be fixed early before they turn into intermittent field failures.