Utility functions for atomic access

This modules adds some utility functions to perform atomic accesses. More...

Detailed Description

This modules adds some utility functions to perform atomic accesses.

Usage

The atomic utilitys allow atomic access to regular variables.

uint32_t global_counter;
 
void irq_handler(void)
{
    // No need to use atomic access in IRQ handlers, if other IRQ handlers
    // never touch global_counter: At the beginning and at the end of every
    // ISR a memory barrier is in place, so that at the end of the ISR the
    // memory will be in a state as if all memory accesses within the ISR
    // took place in sequential order.
    //
    // Extra detail only RIOT kernel hackers need to know: If all ISRs
    // accessing the same variable cannot interrupt each other, atomic
    // access is still not needed. (Currently only PendSV on ARM can be
    // interrupted by other IRQs with RIOTs default IRQ priorities. If
    // application developers modifies those, they can be assumed to know
    // what they are doing - or to happily face the consequences otherwise.)
    global_counter++;
}
 
void called_by_thread_a(void) {
    if (atomic_load_u32(&global_counter) > THRESHOLD) {
        on_threshold_reached();
        atomic_store_u32(&global_counter, 0);
    }
}
 
void called_by_thread_b(void) {
    atomic_add_u32(&global_counter, 42);
}

Motivation

There are some reasons why these functions might be chosen over the C11 Atomic Operations Library in some advanced use cases:

• The functions allow mixing of atomic and non-atomic accesses. E.g. while IRQs are disabled anyway, even plain accesses cannot be interrupted but are often more efficient.
• On platforms not supporting lock-free access, a library call is generated instead. The fallback implementation used here is more efficient in terms of both CPU instructions and ROM size.
• On platforms where some operations can be implemented lock free while others can't, at least LLVM will use the library call even for those accesses that can be implemented lock-free. This is because without assuming how the library call implements atomic access for the other functions, mixing library calls and lock free accesses could result in data corruption. But this implementation resorts to disabling IRQs when lock-free implementations are not possible, which mixes well with lock-free accesses. Thus, additional overhead for atomic accesses is only spent where needed.
• In some cases the fallback implementation performs better than the lock free implementation. E.g. if a specific platform has an atomic compare and swap instruction, this could be used to perform a read-modify-write in a loop until the value initially read was not changed in between. Just disabling IRQs to implement an atomic read-modify-write operation is likely more efficient. C11 atomics will however always use the lock free implementation (if such exists), assuming that this is more efficient. This assumption was made with desktop class hardware in mind, but is not generally true for bare metal targets. These function allow to optimize for the actual hardware RIOT is running on.
• This library provides "semi-atomic" read-modify-write operations, which are useful when at most one thread is ever writing to memory. In that case, only the write part of the read-modify-write operation needs to be performed in an atomic fashion in order for the reading threads to perceive atomic updates of the variable. This is significantly cheaper than atomic read-modify-write operations for many platforms

Guarantees

• Every utility function here acts as a barrier for code reordering regarding
• For the atomic_*() family of functions: The whole operation will be done in an non-interruptible fashion
• For the semi_atomic_*() family of functions: The write part of the operation is done atomically. If at most one thread is ever performing changes to a variable using the semi_atomic_() functions, those changes will appear as if they were atomic to all other threads.

Porting to new CPUs

At the bare minimum, create an empty atomic_utils_arch.h file. This will result in the fallback implementations being used.

To expose lock-free atomic operations, add an implementation to the atomic_utils_arch.h file and disable the fallback implementation by defining HAS_<FN_NAME_ALL_CAPS>, where <FN_NAME_ALL_CAPS> is the name of the function provided in all upper case. E.g. most platforms will be able to provide lock-free reads and writes up to their word size and can expose this as follows for GCC:

// All the user header boilerplate
#define HAS_ATOMIC_LOAD_U8
static inline uint8_t atomic_load_u8(const uint8_t *var)
{
    return __atomic_load_1(var, __ATOMIC_SEQ_CST);
}
 
#define HAS_ATOMIC_STORE_U8
static inline void atomic_store_u8(uint8_t *dest, uint8_t val)
{
    __atomic_store_1(dest, val, __ATOMIC_SEQ_CST);
}

Note: The semi_atomic_*() family of functions is always provided using atomic_*() functions in the cheapest way possible.

Topics
	Helpers for volatile accesses

Files
file	atomic_utils.h
	API of the utility functions for atomic accesses.

Data Structures
struct	atomic_bit_u8_t
	Type specifying a bit in an uint8_t More...
struct	atomic_bit_u16_t
	Type specifying a bit in an uint16_t More...
struct	atomic_bit_u32_t
	Type specifying a bit in an uint32_t More...
struct	atomic_bit_u64_t
	Type specifying a bit in an uint64_t More...

Macros
#define	ATOMIC_LOAD_IMPL(name, type)
	Generates a static inline function implementing atomic_load_u<width>()
#define	ATOMIC_STORE_IMPL(name, type)
	Generates a static inline function implementing atomic_store_u<width>()
#define	ATOMIC_FETCH_OP_IMPL(opname, op, name, type)
	Generates a static inline function implementing atomic_fecth_<op>_u<width>()

Atomic Loads
static uint8_t	atomic_load_u8 (const volatile uint8_t *var)
	Load an uint8_t atomically.
static uint16_t	atomic_load_u16 (const volatile uint16_t *var)
	Load an uint16_t atomically.
static uint32_t	atomic_load_u32 (const volatile uint32_t *var)
	Load an uint32_t atomically.
static uint64_t	atomic_load_u64 (const volatile uint64_t *var)
	Load an uint64_t atomically.
static unsigned	atomic_load_unsigned (const volatile unsigned *var)
	Load an unsigned int atomically.
static uintptr_t	atomic_load_uintptr (const volatile uintptr_t *var)
	Load an uintptr_t atomically.
static void *	atomic_load_ptr (void **ptr_addr)
	Load an void * atomically.
static kernel_pid_t	atomic_load_kernel_pid (const volatile kernel_pid_t *var)
	Load an kernel_pid_t atomically.

Atomic Stores
static void	atomic_store_u8 (volatile uint8_t *dest, uint8_t val)
	Store an uint8_t atomically.
static void	atomic_store_u16 (volatile uint16_t *dest, uint16_t val)
	Store an uint16_t atomically.
static void	atomic_store_u32 (volatile uint32_t *dest, uint32_t val)
	Store an uint32_t atomically.
static void	atomic_store_u64 (volatile uint64_t *dest, uint64_t val)
	Store an uint64_t atomically.
static void	atomic_store_unsigned (volatile unsigned *dest, unsigned val)
	Store an uint64_t atomically.
static void	atomic_store_uintptr (volatile uintptr_t *dest, uintptr_t val)
	Store an uintptr_t atomically.
static void	atomic_store_ptr (void **dest, const void *val)
	Store an void * atomically.
static void	atomic_store_kernel_pid (volatile kernel_pid_t *dest, kernel_pid_t val)
	Store an kernel_pid_t atomically.

Atomic In-Place Addition
static uint8_t	atomic_fetch_add_u8 (volatile uint8_t *dest, uint8_t summand)
	Atomically add a value onto a given value.
static uint16_t	atomic_fetch_add_u16 (volatile uint16_t *dest, uint16_t summand)
	Atomically add a value onto a given value.
static uint32_t	atomic_fetch_add_u32 (volatile uint32_t *dest, uint32_t summand)
	Atomically add a value onto a given value.
static uint64_t	atomic_fetch_add_u64 (volatile uint64_t *dest, uint64_t summand)
	Atomically add a value onto a given value.
static unsigned	atomic_fetch_add_unsigned (volatile unsigned *dest, unsigned summand)
	Atomically add a value onto a given value.

Atomic In-Place Subtraction
static uint8_t	atomic_fetch_sub_u8 (volatile uint8_t *dest, uint8_t subtrahend)
	Atomically subtract a value from a given value.
static uint16_t	atomic_fetch_sub_u16 (volatile uint16_t *dest, uint16_t subtrahend)
	Atomically subtract a value from a given value.
static uint32_t	atomic_fetch_sub_u32 (volatile uint32_t *dest, uint32_t subtrahend)
	Atomically subtract a value from a given value.
static uint64_t	atomic_fetch_sub_u64 (volatile uint64_t *dest, uint64_t subtrahend)
	Atomically subtract a value from a given value.
static unsigned	atomic_fetch_sub_unsigned (volatile unsigned *dest, unsigned subtrahend)
	Atomically subtract a value from a given value.

Atomic In-Place Bitwise OR
static uint8_t	atomic_fetch_or_u8 (volatile uint8_t *dest, uint8_t val)
	Atomic version of *dest |= val
static uint16_t	atomic_fetch_or_u16 (volatile uint16_t *dest, uint16_t val)
	Atomic version of *dest |= val
static uint32_t	atomic_fetch_or_u32 (volatile uint32_t *dest, uint32_t val)
	Atomic version of *dest |= val
static uint64_t	atomic_fetch_or_u64 (volatile uint64_t *dest, uint64_t val)
	Atomic version of *dest |= val
static unsigned	atomic_fetch_or_unsigned (volatile unsigned *dest, unsigned val)
	Atomic version of *dest |= val

Atomic In-Place Bitwise XOR
static uint8_t	atomic_fetch_xor_u8 (volatile uint8_t *dest, uint8_t val)
	Atomic version of *dest ^= val
static uint16_t	atomic_fetch_xor_u16 (volatile uint16_t *dest, uint16_t val)
	Atomic version of *dest ^= val
static uint32_t	atomic_fetch_xor_u32 (volatile uint32_t *dest, uint32_t val)
	Atomic version of *dest ^= val
static uint64_t	atomic_fetch_xor_u64 (volatile uint64_t *dest, uint64_t val)
	Atomic version of *dest ^= val
static unsigned	atomic_fetch_xor_unsigned (volatile unsigned *dest, unsigned val)
	Atomic version of *dest ^= val

Atomic In-Place Bitwise AND
static uint8_t	atomic_fetch_and_u8 (volatile uint8_t *dest, uint8_t val)
	Atomic version of *dest &= val
static uint16_t	atomic_fetch_and_u16 (volatile uint16_t *dest, uint16_t val)
	Atomic version of *dest &= val
static uint32_t	atomic_fetch_and_u32 (volatile uint32_t *dest, uint32_t val)
	Atomic version of *dest &= val
static uint64_t	atomic_fetch_and_u64 (volatile uint64_t *dest, uint64_t val)
	Atomic version of *dest &= val
static unsigned	atomic_fetch_and_unsigned (volatile unsigned *dest, unsigned val)
	Atomic version of *dest &= val

Helper Functions to Handle Atomic Bit References
static atomic_bit_u8_t	atomic_bit_u8 (volatile uint8_t *dest, uint8_t bit)
	Create a reference to a bit in an uint8_t
static atomic_bit_u16_t	atomic_bit_u16 (volatile uint16_t *dest, uint8_t bit)
	Create a reference to a bit in an uint16_t
static atomic_bit_u32_t	atomic_bit_u32 (volatile uint32_t *dest, uint8_t bit)
	Create a reference to a bit in an uint32_t
static atomic_bit_u64_t	atomic_bit_u64 (volatile uint64_t *dest, uint8_t bit)
	Create a reference to a bit in an uint64_t

Atomic Bit Setting
static void	atomic_set_bit_u8 (atomic_bit_u8_t bit)
	Atomic version of *dest |= (1 << bit)
static void	atomic_set_bit_u16 (atomic_bit_u16_t bit)
	Atomic version of *dest |= (1 << bit)
static void	atomic_set_bit_u32 (atomic_bit_u32_t bit)
	Atomic version of *dest |= (1 << bit)
static void	atomic_set_bit_u64 (atomic_bit_u64_t bit)
	Atomic version of *dest |= (1 << bit)

Atomic Bit Clearing
static void	atomic_clear_bit_u8 (atomic_bit_u8_t bit)
	Atomic version of *dest &= ~(1 << bit)
static void	atomic_clear_bit_u16 (atomic_bit_u16_t bit)
	Atomic version of *dest &= ~(1 << bit)
static void	atomic_clear_bit_u32 (atomic_bit_u32_t bit)
	Atomic version of *dest &= ~(1 << bit)
static void	atomic_clear_bit_u64 (atomic_bit_u64_t bit)
	Atomic version of *dest &= ~(1 << bit)

Semi-Atomic In-Place Addition
static uint8_t	semi_atomic_fetch_add_u8 (volatile uint8_t *dest, uint8_t summand)
	Semi-atomically add a value onto a given value.
static uint16_t	semi_atomic_fetch_add_u16 (volatile uint16_t *dest, uint16_t summand)
	Semi-atomically add a value onto a given value.
static uint32_t	semi_atomic_fetch_add_u32 (volatile uint32_t *dest, uint32_t summand)
	Semi-atomically add a value onto a given value.
static uint64_t	semi_atomic_fetch_add_u64 (volatile uint64_t *dest, uint64_t summand)
	Semi-atomically add a value onto a given value.
static unsigned	semi_atomic_fetch_add_unsigned (volatile unsigned *dest, unsigned summand)
	Semi-atomically add a value onto a given value.

Semi-Atomic In-Place Subtraction
static uint8_t	semi_atomic_fetch_sub_u8 (volatile uint8_t *dest, uint8_t subtrahend)
	Semi-atomically subtract a value from a given value.
static uint16_t	semi_atomic_fetch_sub_u16 (volatile uint16_t *dest, uint16_t subtrahend)
	Semi-atomically subtract a value from a given value.
static uint32_t	semi_atomic_fetch_sub_u32 (volatile uint32_t *dest, uint32_t subtrahend)
	Semi-atomically subtract a value from a given value.
static uint64_t	semi_atomic_fetch_sub_u64 (volatile uint64_t *dest, uint64_t subtrahend)
	Semi-atomically subtract a value from a given value.
static unsigned	semi_atomic_fetch_sub_unsigned (volatile unsigned *dest, unsigned subtrahend)
	Semi-atomically subtract a value from a given value.

Semi-atomic In-Place Bitwise OR
static uint8_t	semi_atomic_fetch_or_u8 (volatile uint8_t *dest, uint8_t val)
	Semi-atomic version of *dest |= val
static uint16_t	semi_atomic_fetch_or_u16 (volatile uint16_t *dest, uint16_t val)
	Semi-atomic version of *dest |= val
static uint32_t	semi_atomic_fetch_or_u32 (volatile uint32_t *dest, uint32_t val)
	Semi-atomic version of *dest |= val
static uint64_t	semi_atomic_fetch_or_u64 (volatile uint64_t *dest, uint64_t val)
	Semi-atomic version of *dest |= val
static unsigned	semi_atomic_fetch_or_unsigned (volatile unsigned *dest, unsigned val)
	Semi-atomic version of *dest |= val

Semi-Atomic In-Place Bitwise XOR
static uint8_t	semi_atomic_fetch_xor_u8 (volatile uint8_t *dest, uint8_t val)
	Semi-atomic version of *dest ^= val
static uint16_t	semi_atomic_fetch_xor_u16 (volatile uint16_t *dest, uint16_t val)
	Semi-atomic version of *dest ^= val
static uint32_t	semi_atomic_fetch_xor_u32 (volatile uint32_t *dest, uint32_t val)
	Semi-atomic version of *dest ^= val
static uint64_t	semi_atomic_fetch_xor_u64 (volatile uint64_t *dest, uint64_t val)
	Semi-atomic version of *dest ^= val
static unsigned	semi_atomic_fetch_xor_unsigned (volatile unsigned *dest, unsigned val)
	Semi-atomic version of *dest ^= val

Semi-Atomic In-Place Bitwise AND
static uint8_t	semi_atomic_fetch_and_u8 (volatile uint8_t *dest, uint8_t val)
	Semi-atomic version of *dest &= val
static uint16_t	semi_atomic_fetch_and_u16 (volatile uint16_t *dest, uint16_t val)
	Semi-atomic version of *dest &= val
static uint32_t	semi_atomic_fetch_and_u32 (volatile uint32_t *dest, uint32_t val)
	Semi-atomic version of *dest &= val
static uint64_t	semi_atomic_fetch_and_u64 (volatile uint64_t *dest, uint64_t val)
	Semi-atomic version of *dest &= val
static unsigned	semi_atomic_fetch_and_unsigned (volatile unsigned *dest, unsigned val)
	Semi-atomic version of *dest &= val

Macro Definition Documentation

ATOMIC_FETCH_OP_IMPL

#define ATOMIC_FETCH_OP_IMPL	(		opname,
			op,
			name,
			type )

Value:

    static inline type CONCAT4(atomic_fetch_, opname, _, name)                 \
            (volatile type *dest, type val)                                    \
    {                                                                          \
        unsigned state = irq_disable();                                        \
        const type result = *dest;                                             \
        *dest = result op val;                                                 \
        irq_restore(state);                                                    \
        return result;                                                         \
    }

Generates a static inline function implementing atomic_fecth_<op>_u<width>()

Parameters

opname	Name of the operator in op, e.g. "and" for +
op	Operator to implement atomically, e.g. +
name	Name of the variable type, e.g. "u8"
type	Variable type, e.g. uint8_t

Definition at line 1211 of file atomic_utils.h.

ATOMIC_LOAD_IMPL

#define ATOMIC_LOAD_IMPL	(		name,
			type )

Value:

    static inline type CONCAT(atomic_load_, name)(const volatile type *var)    \
    {                                                                          \
        unsigned state = irq_disable();                                        \
        type result = *var;                                                    \
        irq_restore(state);                                                    \
        return result;                                                         \
    }

Generates a static inline function implementing atomic_load_u<width>()

Parameters

name	Name of the variable type, e.g. "u8"
type	Variable type, e.g. uint8_t

Definition at line 1151 of file atomic_utils.h.

ATOMIC_STORE_IMPL

#define ATOMIC_STORE_IMPL	(		name,
			type )

Value:

    static inline void CONCAT(atomic_store_, name)                             \
            (volatile type *dest, type val)                                    \
    {                                                                          \
        unsigned state = irq_disable();                                        \
        *dest = val;                                                           \
        irq_restore(state);                                                    \
    }

Generates a static inline function implementing atomic_store_u<width>()

Parameters

name	Name of the variable type, e.g. "u8"
type	Variable type, e.g. uint8_t

Definition at line 1180 of file atomic_utils.h.

Function Documentation

atomic_bit_u16()

static atomic_bit_u16_t atomic_bit_u16 ( volatile uint16_t * dest, uint8_t bit )

inlinestatic

Create a reference to a bit in an uint16_t

Parameters

[in]	dest	Memory containing the bit
[in]	bit	Bit number (0 refers to the least significant)

Definition at line 1294 of file atomic_utils.h.

atomic_bit_u32()

static atomic_bit_u32_t atomic_bit_u32 ( volatile uint32_t * dest, uint8_t bit )

inlinestatic

Create a reference to a bit in an uint32_t

Parameters

[in]	dest	Memory containing the bit
[in]	bit	Bit number (0 refers to the least significant)

Definition at line 1300 of file atomic_utils.h.

atomic_bit_u64()

static atomic_bit_u64_t atomic_bit_u64 ( volatile uint64_t * dest, uint8_t bit )

inlinestatic

Create a reference to a bit in an uint64_t

Parameters

[in]	dest	Memory containing the bit
[in]	bit	Bit number (0 refers to the least significant)

Definition at line 1306 of file atomic_utils.h.

atomic_bit_u8()

static atomic_bit_u8_t atomic_bit_u8 ( volatile uint8_t * dest, uint8_t bit )

inlinestatic

Create a reference to a bit in an uint8_t

Parameters

[in]	dest	Memory containing the bit
[in]	bit	Bit number (0 refers to the least significant)

Definition at line 1288 of file atomic_utils.h.

atomic_clear_bit_u16()

static void atomic_clear_bit_u16 ( atomic_bit_u16_t bit )

inlinestatic

Atomic version of *dest &= ~(1 << bit)

Parameters

[in,out]

bit

bit to set

Definition at line 1332 of file atomic_utils.h.

atomic_clear_bit_u32()

static void atomic_clear_bit_u32 ( atomic_bit_u32_t bit )

inlinestatic

Atomic version of *dest &= ~(1 << bit)

Parameters

[in,out]

bit

bit to set

Definition at line 1336 of file atomic_utils.h.

atomic_clear_bit_u64()

static void atomic_clear_bit_u64 ( atomic_bit_u64_t bit )

inlinestatic

Atomic version of *dest &= ~(1 << bit)

Parameters

[in,out]

bit

bit to set

Definition at line 1340 of file atomic_utils.h.

atomic_clear_bit_u8()

static void atomic_clear_bit_u8 ( atomic_bit_u8_t bit )

inlinestatic

Atomic version of *dest &= ~(1 << bit)

Parameters

[in,out]

bit

bit to set

Definition at line 1328 of file atomic_utils.h.

atomic_fetch_add_u16()

static uint16_t atomic_fetch_add_u16 ( volatile uint16_t * dest, uint16_t summand )

inlinestatic

Atomically add a value onto a given value.

Parameters

[in,out]	dest	Add summand onto this value atomically in-place
[in]	summand	Value to add onto dest

Returns

The value previously stored dest

atomic_fetch_add_u32()

static uint32_t atomic_fetch_add_u32 ( volatile uint32_t * dest, uint32_t summand )

inlinestatic

Atomically add a value onto a given value.

Parameters

[in,out]	dest	Add summand onto this value atomically in-place
[in]	summand	Value to add onto dest

Returns

The value previously stored dest

atomic_fetch_add_u64()

static uint64_t atomic_fetch_add_u64 ( volatile uint64_t * dest, uint64_t summand )

inlinestatic

Atomically add a value onto a given value.

Parameters

[in,out]	dest	Add summand onto this value atomically in-place
[in]	summand	Value to add onto dest

Returns

The value previously stored dest

atomic_fetch_add_u8()

static uint8_t atomic_fetch_add_u8 ( volatile uint8_t * dest, uint8_t summand )

inlinestatic

Atomically add a value onto a given value.

Parameters

[in,out]	dest	Add summand onto this value atomically in-place
[in]	summand	Value to add onto dest

Returns

The value previously stored dest

atomic_fetch_add_unsigned()

static unsigned atomic_fetch_add_unsigned ( volatile unsigned * dest, unsigned summand )

inlinestatic

Atomically add a value onto a given value.

Parameters

[in,out]	dest	Add summand onto this value atomically in-place
[in]	summand	Value to add onto dest

Returns

The value previously stored dest

Note

This is effectively an alias of atomic_fetch_add_u64, atomic_fetch_add_u32, or atomic_fetch_add_u16 depending on the size of unsigned int.

Definition at line 448 of file atomic_utils.h.

atomic_fetch_and_u16()

static uint16_t atomic_fetch_and_u16 ( volatile uint16_t * dest, uint16_t val )

inlinestatic

Atomic version of *dest &= val

Parameters

[in,out]	dest	Replace this value with the result of *dest & val
[in]	val	Value to bitwise and into dest in-place

Returns

The value previously stored dest

atomic_fetch_and_u32()

static uint32_t atomic_fetch_and_u32 ( volatile uint32_t * dest, uint32_t val )

inlinestatic

Atomic version of *dest &= val

Parameters

[in,out]	dest	Replace this value with the result of *dest & val
[in]	val	Value to bitwise and into dest in-place

Returns

The value previously stored dest

atomic_fetch_and_u64()

static uint64_t atomic_fetch_and_u64 ( volatile uint64_t * dest, uint64_t val )

inlinestatic

Atomic version of *dest &= val

Parameters

[in,out]	dest	Replace this value with the result of *dest & val
[in]	val	Value to bitwise and into dest in-place

Returns

The value previously stored dest

atomic_fetch_and_u8()

static uint8_t atomic_fetch_and_u8 ( volatile uint8_t * dest, uint8_t val )

inlinestatic

Atomic version of *dest &= val

Parameters

[in,out]	dest	Replace this value with the result of *dest & val
[in]	val	Value to bitwise and into dest in-place

Returns

The value previously stored dest

atomic_fetch_and_unsigned()

static unsigned atomic_fetch_and_unsigned ( volatile unsigned * dest, unsigned val )

inlinestatic

Atomic version of *dest &= val

Parameters

[in,out]	dest	Replace this value with the result of *dest & val
[in]	val	Value to bitwise and into dest in-place

Returns

The value previously stored dest

Note

This is effectively an alias of atomic_fetch_and_u64, atomic_fetch_and_u32, and atomic_fetch_and_u16 depending on the size of unsigned int.

Definition at line 709 of file atomic_utils.h.

atomic_fetch_or_u16()

static uint16_t atomic_fetch_or_u16 ( volatile uint16_t * dest, uint16_t val )

inlinestatic

Atomic version of *dest |= val

Parameters

[in,out]	dest	Replace this value with the result of *dest | val
[in]	val	Value to bitwise or into dest in-place

Returns

The value previously stored dest

atomic_fetch_or_u32()

static uint32_t atomic_fetch_or_u32 ( volatile uint32_t * dest, uint32_t val )

inlinestatic

Atomic version of *dest |= val

Parameters

[in,out]	dest	Replace this value with the result of *dest | val
[in]	val	Value to bitwise or into dest in-place

Returns

The value previously stored dest

atomic_fetch_or_u64()

static uint64_t atomic_fetch_or_u64 ( volatile uint64_t * dest, uint64_t val )

inlinestatic

Atomic version of *dest |= val

Parameters

[in,out]	dest	Replace this value with the result of *dest | val
[in]	val	Value to bitwise or into dest in-place

Returns

The value previously stored dest

atomic_fetch_or_u8()

static uint8_t atomic_fetch_or_u8 ( volatile uint8_t * dest, uint8_t val )

inlinestatic

Atomic version of *dest |= val

Parameters

[in,out]	dest	Replace this value with the result of *dest | val
[in]	val	Value to bitwise or into dest in-place

Returns

The value previously stored dest

atomic_fetch_or_unsigned()

static unsigned atomic_fetch_or_unsigned ( volatile unsigned * dest, unsigned val )

inlinestatic

Atomic version of *dest |= val

Parameters

[in,out]	dest	Replace this value with the result of *dest | val
[in]	val	Value to bitwise or into dest in-place

Returns

The value previously stored dest

Note

This is effectively an alias of atomic_fetch_or_u64, atomic_fetch_or_u32, or atomic_fetch_or_u16 depending on the size of unsigned int.

Definition at line 579 of file atomic_utils.h.

atomic_fetch_sub_u16()

static uint16_t atomic_fetch_sub_u16 ( volatile uint16_t * dest, uint16_t subtrahend )

inlinestatic

Atomically subtract a value from a given value.

Parameters

[in,out]	dest	Subtract subtrahend from this value atomically in-place
[in]	subtrahend	Value to subtract from dest

Returns

The value previously stored dest

atomic_fetch_sub_u32()

static uint32_t atomic_fetch_sub_u32 ( volatile uint32_t * dest, uint32_t subtrahend )

inlinestatic

Atomically subtract a value from a given value.

Parameters

[in,out]	dest	Subtract subtrahend from this value atomically in-place
[in]	subtrahend	Value to subtract from dest

Returns

The value previously stored dest

atomic_fetch_sub_u64()

static uint64_t atomic_fetch_sub_u64 ( volatile uint64_t * dest, uint64_t subtrahend )

inlinestatic

Atomically subtract a value from a given value.

Parameters

[in,out]	dest	Subtract subtrahend from this value atomically in-place
[in]	subtrahend	Value to subtract from dest

Returns

The value previously stored dest

atomic_fetch_sub_u8()

static uint8_t atomic_fetch_sub_u8 ( volatile uint8_t * dest, uint8_t subtrahend )

inlinestatic

Atomically subtract a value from a given value.

Parameters

[in,out]	dest	Subtract subtrahend from this value atomically in-place
[in]	subtrahend	Value to subtract from dest

Returns

The value previously stored dest

atomic_fetch_sub_unsigned()

static unsigned atomic_fetch_sub_unsigned ( volatile unsigned * dest, unsigned subtrahend )

inlinestatic

Atomically subtract a value from a given value.

Parameters

[in,out]	dest	Subtract subtrahend from this value atomically in-place
[in]	subtrahend	Value to subtract from dest

Returns

The value previously stored dest

Note

This is effectively an alias of atomic_fetch_sub_u64, atomic_fetch_sub_u32, or atomic_fetch_sub_u16 depending on the size of unsigned int.

Definition at line 514 of file atomic_utils.h.

atomic_fetch_xor_u16()

static uint16_t atomic_fetch_xor_u16 ( volatile uint16_t * dest, uint16_t val )

inlinestatic

Atomic version of *dest ^= val

Parameters

[in,out]	dest	Replace this value with the result of *dest ^ val
[in]	val	Value to bitwise xor into dest in-place

Returns

The value previously stored dest

atomic_fetch_xor_u32()

static uint32_t atomic_fetch_xor_u32 ( volatile uint32_t * dest, uint32_t val )

inlinestatic

Atomic version of *dest ^= val

Parameters

[in,out]	dest	Replace this value with the result of *dest ^ val
[in]	val	Value to bitwise xor into dest in-place

Returns

The value previously stored dest

atomic_fetch_xor_u64()

static uint64_t atomic_fetch_xor_u64 ( volatile uint64_t * dest, uint64_t val )

inlinestatic

Atomic version of *dest ^= val

Parameters

[in,out]	dest	Replace this value with the result of *dest ^ val
[in]	val	Value to bitwise xor into dest in-place

Returns

The value previously stored dest

atomic_fetch_xor_u8()

static uint8_t atomic_fetch_xor_u8 ( volatile uint8_t * dest, uint8_t val )

inlinestatic

Atomic version of *dest ^= val

Parameters

[in,out]	dest	Replace this value with the result of *dest ^ val
[in]	val	Value to bitwise xor into dest in-place

Returns

The value previously stored dest

atomic_fetch_xor_unsigned()

static unsigned atomic_fetch_xor_unsigned ( volatile unsigned * dest, unsigned val )

inlinestatic

Atomic version of *dest ^= val

Parameters

[in,out]	dest	Replace this value with the result of *dest ^ val
[in]	val	Value to bitwise xor into dest in-place

Returns

The value previously stored dest

Note

This is effectively an alias of atomic_fetch_xor_u64, atomic_fetch_xor_u32, xor atomic_fetch_xor_u16 depending on the size of unsigned int.

Definition at line 644 of file atomic_utils.h.

atomic_load_kernel_pid()

static kernel_pid_t atomic_load_kernel_pid ( const volatile kernel_pid_t * var )

inlinestatic

Load an kernel_pid_t atomically.

Parameters

[in]

var

Variable to load atomically

Returns

The value stored in var

Definition at line 306 of file atomic_utils.h.

atomic_load_ptr()

static void * atomic_load_ptr ( void ** ptr_addr )

inlinestatic

Load an void * atomically.

Parameters

[in]

ptr_addr

Address to the pointer to load

Returns

Value of the loaded pointer

Definition at line 297 of file atomic_utils.h.

atomic_load_u16()

static uint16_t atomic_load_u16 ( const volatile uint16_t * var )

inlinestatic

Load an uint16_t atomically.

Parameters

[in]

var

Variable to load atomically

Returns

The value stored in var

atomic_load_u32()

static uint32_t atomic_load_u32 ( const volatile uint32_t * var )

inlinestatic

Load an uint32_t atomically.

Parameters

[in]

var

Variable to load atomically

Returns

The value stored in var

atomic_load_u64()

static uint64_t atomic_load_u64 ( const volatile uint64_t * var )

inlinestatic

Load an uint64_t atomically.

Parameters

[in]

var

Variable to load atomically

Returns

The value stored in var

atomic_load_u8()

static uint8_t atomic_load_u8 ( const volatile uint8_t * var )

inlinestatic

Load an uint8_t atomically.

Parameters

[in]

var

Variable to load atomically

Returns

The value stored in var

atomic_load_uintptr()

static uintptr_t atomic_load_uintptr ( const volatile uintptr_t * var )

inlinestatic

Load an uintptr_t atomically.

Parameters

[in]

var

Variable to load atomically

Returns

The value stored in var

Definition at line 280 of file atomic_utils.h.

atomic_load_unsigned()

static unsigned atomic_load_unsigned ( const volatile unsigned * var )

inlinestatic

Load an unsigned int atomically.

Parameters

[in]

var

Variable to load atomically

Returns

The value stored in var

Note

This effectively an alias for atomic_load_u64, atomic_load_u32, or atomic_load_u16 depending on the size of unsigned int.

Definition at line 261 of file atomic_utils.h.

atomic_set_bit_u16()

static void atomic_set_bit_u16 ( atomic_bit_u16_t bit )

inlinestatic

Atomic version of *dest |= (1 << bit)

Parameters

[in,out]

bit

bit to set

Definition at line 1316 of file atomic_utils.h.

atomic_set_bit_u32()

static void atomic_set_bit_u32 ( atomic_bit_u32_t bit )

inlinestatic

Atomic version of *dest |= (1 << bit)

Parameters

[in,out]

bit

bit to set

Definition at line 1320 of file atomic_utils.h.

atomic_set_bit_u64()

static void atomic_set_bit_u64 ( atomic_bit_u64_t bit )

inlinestatic

Atomic version of *dest |= (1 << bit)

Parameters

[in,out]

bit

bit to set

Definition at line 1324 of file atomic_utils.h.

atomic_set_bit_u8()

static void atomic_set_bit_u8 ( atomic_bit_u8_t bit )

inlinestatic

Atomic version of *dest |= (1 << bit)

Parameters

[in,out]

bit

bit to set

Definition at line 1312 of file atomic_utils.h.

atomic_store_kernel_pid()

static void atomic_store_kernel_pid ( volatile kernel_pid_t * dest, kernel_pid_t val )

inlinestatic

Store an kernel_pid_t atomically.

Parameters

[out]	dest	Location to atomically write the new value to
[in]	val	Value to write

Definition at line 395 of file atomic_utils.h.

atomic_store_ptr()

static void atomic_store_ptr ( void ** dest, const void * val )

inlinestatic

Store an void * atomically.

Parameters

[out]	dest	Location to atomically write the new value to
[in]	val	Value to write

Definition at line 386 of file atomic_utils.h.

atomic_store_u16()

static void atomic_store_u16 ( volatile uint16_t * dest, uint16_t val )

inlinestatic

Store an uint16_t atomically.

Parameters

[out]	dest	Location to atomically write the new value to
[in]	val	Value to write

atomic_store_u32()

static void atomic_store_u32 ( volatile uint32_t * dest, uint32_t val )

inlinestatic

Store an uint32_t atomically.

Parameters

[out]	dest	Location to atomically write the new value to
[in]	val	Value to write

atomic_store_u64()

static void atomic_store_u64 ( volatile uint64_t * dest, uint64_t val )

inlinestatic

Store an uint64_t atomically.

Parameters

[out]	dest	Location to atomically write the new value to
[in]	val	Value to write

atomic_store_u8()

static void atomic_store_u8 ( volatile uint8_t * dest, uint8_t val )

inlinestatic

Store an uint8_t atomically.

Parameters

[out]	dest	Location to atomically write the new value to
[in]	val	Value to write

atomic_store_uintptr()

static void atomic_store_uintptr ( volatile uintptr_t * dest, uintptr_t val )

inlinestatic

Store an uintptr_t atomically.

Parameters

[out]	dest	Location to atomically write the new value to
[in]	val	Value to write

Definition at line 368 of file atomic_utils.h.

atomic_store_unsigned()

static void atomic_store_unsigned ( volatile unsigned * dest, unsigned val )

inlinestatic

Store an uint64_t atomically.

Parameters

[out]	dest	Location to atomically write the new value to
[in]	val	Value to write

Note

This is effectively an alias of atomic_store_u64, atomic_store_u32, or atomic_store_u16 depending on the size of unsigned int.

Definition at line 349 of file atomic_utils.h.

semi_atomic_fetch_add_u16()

static uint16_t semi_atomic_fetch_add_u16 ( volatile uint16_t * dest, uint16_t summand )

inlinestatic

Semi-atomically add a value onto a given value.

Parameters

[in,out]	dest	Add summand onto this value semi-atomically in-place
[in]	summand	Value to add onto dest

Returns

The value previously stored dest

Definition at line 1376 of file atomic_utils.h.

semi_atomic_fetch_add_u32()

static uint32_t semi_atomic_fetch_add_u32 ( volatile uint32_t * dest, uint32_t summand )

inlinestatic

Semi-atomically add a value onto a given value.

Parameters

[in,out]	dest	Add summand onto this value semi-atomically in-place
[in]	summand	Value to add onto dest

Returns

The value previously stored dest

Definition at line 1392 of file atomic_utils.h.

semi_atomic_fetch_add_u64()

static uint64_t semi_atomic_fetch_add_u64 ( volatile uint64_t * dest, uint64_t summand )

inlinestatic

Semi-atomically add a value onto a given value.

Parameters

[in,out]	dest	Add summand onto this value semi-atomically in-place
[in]	summand	Value to add onto dest

Returns

The value previously stored dest

Definition at line 1408 of file atomic_utils.h.

semi_atomic_fetch_add_u8()

static uint8_t semi_atomic_fetch_add_u8 ( volatile uint8_t * dest, uint8_t summand )

inlinestatic

Semi-atomically add a value onto a given value.

Parameters

[in,out]	dest	Add summand onto this value semi-atomically in-place
[in]	summand	Value to add onto dest

Returns

The value previously stored dest

Definition at line 1360 of file atomic_utils.h.

semi_atomic_fetch_add_unsigned()

static unsigned semi_atomic_fetch_add_unsigned ( volatile unsigned * dest, unsigned summand )

inlinestatic

Semi-atomically add a value onto a given value.

Parameters

[in,out]	dest	Add summand onto this value semi-atomically in-place
[in]	summand	Value to add onto dest

Returns

The value previously stored dest

Note

This is effectively an alias of semi_atomic_fetch_add_u64, semi_atomic_fetch_add_u32, or semi_atomic_fetch_add_u16, depending on the size of unsigned int.

Definition at line 864 of file atomic_utils.h.

semi_atomic_fetch_and_u16()

static uint16_t semi_atomic_fetch_and_u16 ( volatile uint16_t * dest, uint16_t val )

inlinestatic

Semi-atomic version of *dest &= val

Parameters

[in,out]	dest	Replace this value with the result of *dest & val
[in]	val	Value to bitwise and into dest in-place

Returns

The value previously stored dest

Definition at line 1634 of file atomic_utils.h.

semi_atomic_fetch_and_u32()

static uint32_t semi_atomic_fetch_and_u32 ( volatile uint32_t * dest, uint32_t val )

inlinestatic

Semi-atomic version of *dest &= val

Parameters

[in,out]	dest	Replace this value with the result of *dest & val
[in]	val	Value to bitwise and into dest in-place

Returns

The value previously stored dest

Definition at line 1650 of file atomic_utils.h.

semi_atomic_fetch_and_u64()

static uint64_t semi_atomic_fetch_and_u64 ( volatile uint64_t * dest, uint64_t val )

inlinestatic

Semi-atomic version of *dest &= val

Parameters

[in,out]	dest	Replace this value with the result of *dest & val
[in]	val	Value to bitwise and into dest in-place

Returns

The value previously stored dest

Definition at line 1666 of file atomic_utils.h.

semi_atomic_fetch_and_u8()

static uint8_t semi_atomic_fetch_and_u8 ( volatile uint8_t * dest, uint8_t val )

inlinestatic

Semi-atomic version of *dest &= val

Parameters

[in,out]	dest	Replace this value with the result of *dest & val
[in]	val	Value to bitwise and into dest in-place

Returns

The value previously stored dest

Definition at line 1618 of file atomic_utils.h.

semi_atomic_fetch_and_unsigned()

static unsigned semi_atomic_fetch_and_unsigned ( volatile unsigned * dest, unsigned val )

inlinestatic

Semi-atomic version of *dest &= val

Parameters

[in,out]	dest	Replace this value with the result of *dest & val
[in]	val	Value to bitwise and into dest in-place

Returns

The value previously stored dest

Note

This is effectively an alias of semi_atomic_fetch_and_u64, semi_atomic_fetch_and_u32, and semi_atomic_fetch_and_u16, depending on the size of unsigned int.

Definition at line 1127 of file atomic_utils.h.

semi_atomic_fetch_or_u16()

static uint16_t semi_atomic_fetch_or_u16 ( volatile uint16_t * dest, uint16_t val )

inlinestatic

Semi-atomic version of *dest |= val

Parameters

[in,out]	dest	Replace this value with the result of *dest | val
[in]	val	Value to bitwise or into dest in-place

Returns

The value previously stored dest

Definition at line 1504 of file atomic_utils.h.

semi_atomic_fetch_or_u32()

static uint32_t semi_atomic_fetch_or_u32 ( volatile uint32_t * dest, uint32_t val )

inlinestatic

Semi-atomic version of *dest |= val

Parameters

[in,out]	dest	Replace this value with the result of *dest | val
[in]	val	Value to bitwise or into dest in-place

Returns

The value previously stored dest

Definition at line 1520 of file atomic_utils.h.

semi_atomic_fetch_or_u64()

static uint64_t semi_atomic_fetch_or_u64 ( volatile uint64_t * dest, uint64_t val )

inlinestatic

Semi-atomic version of *dest |= val

Parameters

[in,out]	dest	Replace this value with the result of *dest | val
[in]	val	Value to bitwise or into dest in-place

Returns

The value previously stored dest

Definition at line 1536 of file atomic_utils.h.

semi_atomic_fetch_or_u8()

static uint8_t semi_atomic_fetch_or_u8 ( volatile uint8_t * dest, uint8_t val )

inlinestatic

Semi-atomic version of *dest |= val

Parameters

[in,out]	dest	Replace this value with the result of *dest | val
[in]	val	Value to bitwise or into dest in-place

Returns

The value previously stored dest

Definition at line 1488 of file atomic_utils.h.

semi_atomic_fetch_or_unsigned()

static unsigned semi_atomic_fetch_or_unsigned ( volatile unsigned * dest, unsigned val )

inlinestatic

Semi-atomic version of *dest |= val

Parameters

[in,out]	dest	Replace this value with the result of *dest | val
[in]	val	Value to bitwise or into dest in-place

Returns

The value previously stored dest

Note

This is effectively an alias of semi_atomic_fetch_or_u64, semi_atomic_fetch_or_u32, or semi_atomic_fetch_or_u16, depending on the size of unsigned int.

Definition at line 995 of file atomic_utils.h.

semi_atomic_fetch_sub_u16()

static uint16_t semi_atomic_fetch_sub_u16 ( volatile uint16_t * dest, uint16_t subtrahend )

inlinestatic

Semi-atomically subtract a value from a given value.

Parameters

[in,out]	dest	Subtract subtrahend from this value semi-atomically in-place
[in]	subtrahend	Value to subtract from dest

Returns

The value previously stored dest

Definition at line 1439 of file atomic_utils.h.

semi_atomic_fetch_sub_u32()

static uint32_t semi_atomic_fetch_sub_u32 ( volatile uint32_t * dest, uint32_t subtrahend )

inlinestatic

Semi-atomically subtract a value from a given value.

Parameters

[in,out]	dest	Subtract subtrahend from this value semi-atomically in-place
[in]	subtrahend	Value to subtract from dest

Returns

The value previously stored dest

Definition at line 1455 of file atomic_utils.h.

semi_atomic_fetch_sub_u64()

static uint64_t semi_atomic_fetch_sub_u64 ( volatile uint64_t * dest, uint64_t subtrahend )

inlinestatic

Semi-atomically subtract a value from a given value.

Parameters

[in,out]	dest	Subtract subtrahend from this value semi-atomically in-place
[in]	subtrahend	Value to subtract from dest

Returns

The value previously stored dest

Definition at line 1471 of file atomic_utils.h.

semi_atomic_fetch_sub_u8()

static uint8_t semi_atomic_fetch_sub_u8 ( volatile uint8_t * dest, uint8_t subtrahend )

inlinestatic

Semi-atomically subtract a value from a given value.

Parameters

[in,out]	dest	Subtract subtrahend from this value semi-atomically in-place
[in]	subtrahend	Value to subtract from dest

Returns

The value previously stored dest

Definition at line 1423 of file atomic_utils.h.

semi_atomic_fetch_sub_unsigned()

static unsigned semi_atomic_fetch_sub_unsigned ( volatile unsigned * dest, unsigned subtrahend )

inlinestatic

Semi-atomically subtract a value from a given value.

Parameters

[in,out]	dest	Subtract subtrahend from this value semi-atomically in-place
[in]	subtrahend	Value to subtract from dest

Returns

The value previously stored dest

Note

This is effectively an alias of semi_atomic_fetch_sub_u64, semi_atomic_fetch_sub_u32, or semi_atomic_fetch_sub_u16, depending on the size of unsigned int.

Definition at line 930 of file atomic_utils.h.

semi_atomic_fetch_xor_u16()

static uint16_t semi_atomic_fetch_xor_u16 ( volatile uint16_t * dest, uint16_t val )

inlinestatic

Semi-atomic version of *dest ^= val

Parameters

[in,out]	dest	Replace this value with the result of *dest ^ val
[in]	val	Value to bitwise xor into dest in-place

Returns

The value previously stored dest

Definition at line 1569 of file atomic_utils.h.

semi_atomic_fetch_xor_u32()

static uint32_t semi_atomic_fetch_xor_u32 ( volatile uint32_t * dest, uint32_t val )

inlinestatic

Semi-atomic version of *dest ^= val

Parameters

[in,out]	dest	Replace this value with the result of *dest ^ val
[in]	val	Value to bitwise xor into dest in-place

Returns

The value previously stored dest

Definition at line 1585 of file atomic_utils.h.

semi_atomic_fetch_xor_u64()

static uint64_t semi_atomic_fetch_xor_u64 ( volatile uint64_t * dest, uint64_t val )

inlinestatic

Semi-atomic version of *dest ^= val

Parameters

[in,out]	dest	Replace this value with the result of *dest ^ val
[in]	val	Value to bitwise xor into dest in-place

Returns

The value previously stored dest

Definition at line 1601 of file atomic_utils.h.

semi_atomic_fetch_xor_u8()

static uint8_t semi_atomic_fetch_xor_u8 ( volatile uint8_t * dest, uint8_t val )

inlinestatic

Semi-atomic version of *dest ^= val

Parameters

[in,out]	dest	Replace this value with the result of *dest ^ val
[in]	val	Value to bitwise xor into dest in-place

Returns

The value previously stored dest

Definition at line 1553 of file atomic_utils.h.

semi_atomic_fetch_xor_unsigned()

static unsigned semi_atomic_fetch_xor_unsigned ( volatile unsigned * dest, unsigned val )

inlinestatic

Semi-atomic version of *dest ^= val

Parameters

[in,out]	dest	Replace this value with the result of *dest ^ val
[in]	val	Value to bitwise xor into dest in-place

Returns

The value previously stored dest

Note

This is effectively an alias of semi_atomic_fetch_xor_u64, semi_atomic_fetch_xor_u32, xor semi_atomic_fetch_xor_u16, depending on the size of unsigned int.

Definition at line 1061 of file atomic_utils.h.