llfio_v2_xxx::utils Namespace Reference

Utility routines often useful when using LLFIO. More...

Classes
class	page_allocator
	An STL allocator which allocates large TLB page memory. More...
class	page_allocator< void >
struct	process_memory_usage
	Memory usage statistics for a process. More...

Functions
size_t	page_size () noexcept
	Returns the smallest page size of this architecture which is useful for calculating direct i/o multiples. More...
template<class T >
T	round_down_to_page_size (T i, size_t pagesize) noexcept
	Round a value to its next lowest page size multiple.
template<class T >
T	round_up_to_page_size (T i, size_t pagesize) noexcept
	Round a value to its next highest page size multiple.
LLFIO_TEMPLATE(class T) LLFIO_TREQUIRES(LLFIO_TEXPR(std size_t	file_buffer_default_size ()
	Round a pair of a pointer and a size_t to their nearest page size multiples. The pointer will be rounded down, the size_t upwards. More...
void	random_fill (char *buffer, size_t bytes) noexcept
	Fills the buffer supplied with cryptographically strong randomness. Uses the OS kernel API. More...
std::string	random_string (size_t randomlen)
	Returns a cryptographically random string capable of being used as a filename. Essentially random_fill() + to_hex_string(). More...
result< void >	flush_modified_data () noexcept
	Tries to flush all modified data to the physical device.
result< void >	drop_filesystem_cache () noexcept
	Tries to flush all modified data to the physical device, and then drop the OS filesystem cache, thus making all future reads come from the physical device. Currently only implemented for Microsoft Windows and Linux. More...
bool	running_under_wsl () noexcept
	Returns true if this POSIX is running under Microsoft's Subsystem for Linux.
result< process_memory_usage >	current_process_memory_usage () noexcept
	Retrieve the current memory usage statistics for this process. More...
template<class T , class U >
bool	operator== (const page_allocator< T > &, const page_allocator< U > &) noexcept

Detailed Description

Utility routines often useful when using LLFIO.

Function Documentation

current_process_memory_usage()

result<process_memory_usage> llfio_v2_xxx::utils::current_process_memory_usage ( )

inlinenoexcept

Retrieve the current memory usage statistics for this process.

Note

Mac OS provides no way of reading how much memory a process has committed. We therefore supply as private_committed the same value as private_paged_in.

drop_filesystem_cache()

result<void> llfio_v2_xxx::utils::drop_filesystem_cache ( )

inlinenoexcept

Tries to flush all modified data to the physical device, and then drop the OS filesystem cache, thus making all future reads come from the physical device. Currently only implemented for Microsoft Windows and Linux.

Note that the OS specific magic called by this routine generally requires elevated privileges for the calling process. For obvious reasons, calling this will have a severe negative impact on performance, but it's very useful for benchmarking cold cache vs warm cache performance.

file_buffer_default_size()

LLFIO_TEMPLATE (class T) LLFIO_TREQUIRES(LLFIO_TEXPR(std size_t llfio_v2_xxx::utils::file_buffer_default_size ( )

inline

Round a pair of a pointer and a size_t to their nearest page size multiples. The pointer will be rounded down, the size_t upwards.

Round a pair of a pointer and a size_t to their nearest page size multiples. The pointer will be rounded upwards, the size_t downwards.

Round a pair of values to their nearest page size multiples. The first will be rounded down, the second upwards.

Round a pair of values to their nearest page size multiples. The first will be rounded upwards, the second downwards.

Returns the page sizes of this architecture which is useful for calculating direct i/o multiples.

Parameters

only_actually_available

Only return page sizes actually available to the user running this process

Returns

The page sizes of this architecture.

Complexity\nFirst call performs multiple memory allocations, mutex locks and system calls. Subsequent calls

lock mutexes.

Errors returnable\nThrows any error from the operating system or std::bad_alloc.

Returns a reasonable default size for page_allocator, typically the closest page size from page_sizes() to 1Mb.

Returns

A value of a TLB large page size close to 1Mb.

Complexity\nWhatever the system API takes (one would hope constant time).

Errors returnable\nThrows any error from the operating system or std::bad_alloc.

  {
    static size_t size;
    if(size == 0u)
    {
      const std::vector<size_t> &sizes = page_sizes(true);
      for(auto &i : sizes)
      {
        if(i >= 1024 * 1024)
        {
          size = i;
          break;
        }
      }
      if(size == 0u)
      {
        size = 1024 * 1024;
      }
    }
    return size;
  }

page_size()

size_t llfio_v2_xxx::utils::page_size ( )

inlinenoexcept

Returns the smallest page size of this architecture which is useful for calculating direct i/o multiples.

Returns

The page size of this architecture.

Complexity\nWhatever the system API takes (one would hope constant time).

random_fill()

void llfio_v2_xxx::utils::random_fill ( char *  buffer, size_t  bytes  )

inlinenoexcept

Fills the buffer supplied with cryptographically strong randomness. Uses the OS kernel API.

Parameters

buffer	A buffer to fill
bytes	How many bytes to fill

Complexity\nWhatever the system API takes.

Errors returnable\nAny error from the operating system.

random_string()

std::string llfio_v2_xxx::utils::random_string ( size_t  randomlen )

inline

Returns a cryptographically random string capable of being used as a filename. Essentially random_fill() + to_hex_string().

Parameters

randomlen

The number of bytes of randomness to use for the string.

Returns

A string representing the randomness at a 2x ratio, so if 32 bytes were requested, this string would be 64 bytes long.

Complexity\nWhatever the system API takes.

Errors returnable\nAny error from the operating system.

  {
    size_t outlen = randomlen * 2;
    std::string ret(outlen, 0);
    random_fill(const_cast<char *>(ret.data()), randomlen);
    QUICKCPPLIB_NAMESPACE::algorithm::string::to_hex_string(const_cast<char *>(ret.data()), outlen, ret.data(), randomlen);
    return ret;
  }