WOLFSSL STATIC BUFFER ALLOCATION OPTION

Static Buffer Allocation Option

Documentation and User’s Guide

May 2016 (revised June 2017)


1.0 INTRO

This document covers the API and build of wolfSSL with the staticmemory feature. Enabling the feature allows for use of static memory when performing a TLS/DTLS connection. To avoid calling system free’s on static memory and avoid memory not getting returned to a heap as available, the user should never call free on values created from wolfSSL functions in static memory mode. If a case is encountered where calling free is necessary on objects created by and returned from a wolfSSL function call, please contact us at wolfSSL for handling the free.

2.0 INSTALLATION

To enable the use of static memory with wolfSSL connections the configure flag staticmemory is used. An example of building wolfSSL with it enabled is as follows.

./configure --enable-staticmemory

Use of the static memory feature requires that WOLFSSL_SMALL_STACK not be turned on and that USE_FAST_MATH be defined. By default autotools turns on fastmath on 64bit systems. For non 64bit systems use --enable-fastmath, on some 32bit systems clang will be needed or C_EXTRA_FLAGS=-DTMF_NO_ASM for building with fastmath. For builds not using autotools, settings.h has a compile time error message displayed if these conditions are not met.

3.0 Design Overview


Static memory is assigned to a CTX structure that then passes a heap hint to all structures created by the CTX. In the following diagram the arrow indicates passing a pointer to the heap, “…” shows is all structs and not specific to the ones listed.

Static Buffer Allocation
This allows for a connection established using a particular CTX to use the heap assigned to it. Functions being called with not using a structure that has a heap hint will not use the static memory set aside.
Memory is divided into to two separate types; IO memory, and general use. Depending on which flag is used when loading in memory will determine which type it is. By default it is stored as general memory. Assigning memory as IO memory using the flag WOLFMEM_IO_POOL will keep it set aside for when trying to malloc memory for use with sending and receiving information across the connection. This is to eliminate competition with none IO processes. Each IO pool of memory is close to 17k, this is to account for the max TLS packet size being 16k with some overhead.
There are some current function restrictions due to heap hint being passed in. These restrictions are derived from the design of passing heap hint from structure to structure since the functions have no connection with the base CTX but use dynamic memory. Many of these functions are in the OpenSSL compatibility layer. All of these can be ran with the default --enable-staticmemory but in cases will be making calls to malloc instead of using the static memory heap hint.

Functions for creating CTX:
wolfSSL_CTX_load_static_memory() should be used to create a static ctx instead of wolfSSL_CTX_new(). wolfSSL_CTX_new() can be used but will use dynamic memory for the creation of ctx and for the creation of WOLFSSL_METHOD struct. To avoid using dynamic memory wolfSSL_CTX_load_static_memory() can take a ctx set as NULL and a static method function (ending in _ex) to use only memory from the static buffer.

Functions that will not malloc in static mode and return null: these functions have the option of taking in a buffer or creating a buffer. If a buffer is not passed in as an argument then when using static memory a NULL value is returned.

char* wolfSSL_X509_NAME_oneline

byte* wolfSSL_X509_get_device_type

byte* wolfSSL_X509_get_hw_type

byte* wolfSSL_X509_get_hw_serial_number

Functions not supported to use static memory: these functions call malloc without any connection to a ctx heap hint. Many are in the OpenSSL compatibility layer and have the parameters and behavior they do to allow compatibility.

WOLFSSL_X509* wolfSSL_X509_d2i

WOLFSSL_X509* wolfSSL_X509_d2i_fp

WOLFSSL_X509* wolfSSL_X509_load_certificate_file

WOLFSSL_X509_STORE* wolfSSL_X509_STORE_new

WOLFSSL_X509_STORE_CTX* wolfSSL_X509_STORE_CTX_new

WOLFSSL_BIGNUM* wolfSSL_BN_new

void wolfSSL_BN_free

char *wolfSSL_BN_bn2dec

char *wolfSSL_BN_bn2hex

WOLFSSL_DH* wolfSSL_DH_new

WOLFSSL_DSA* wolfSSL_DSA_new

WOLFSSL_RSA* wolfSSL_RSA_new

int wolfSSL_PEM_write_mem_ECPrivateKey

int wolfSSL_PEM_write_mem_DSAPrivateKey

int wolfSSL_PEM_write_mem_RSAPrivateKey

WOLFSSL_EC_KEY *wolfSSL_EC_KEY_new

WOLFSSL_EC_GROUP *wolfSSL_EC_GROUP_new_by_curve_name

WOLFSSL_ECDSA_SIG *wolfSSL_ECDSA_SIG_new

int wolfSSL_PEM_write_mem_ECPrivateKey

int wolfSSL_PEM_write_mem_DSAPrivateKey

WOLFSSL_X509* wolfSSL_get_chain_X509

WOLFSSL_BIO* wolfSSL_BIO_new_socket

WOLFSSL_BIO* wolfSSL_BIO_new

WOLFSSL_BIO* wolfSSL_BIO_new_mem_buf

These 2 have fixed static buffer sizes by default and if a file is larger it is then dynamic: with these two functions there is partial functionality with static buffers. A buffer is placed on the stack by default and if not large enough then the buffer is increased using dynamic memory. In the case of static memory a MEMORY_E value is returned for files too large with these functions rather than creating new dynamic memory.

int wolfSSL_PemCertToDer

int wolfSSL_PemPubKeyToDer

If absolutely no mallocs is sought after then WOLFSSL_NO_MALLOC can be defined. This macro will have wolfSSL not compile in the use of malloc when a heap hint is not used. It will also cause make check to fail with unit tests and api tests that do not have heap hints. With WOLFSSL_NO_MALLOC examples/server/server is setup to function and can be ran to show a connection with absolutely no application mallocs. (note: using something like iprofiler will show system mallocs such as fopen for when certificate files are opened)

 

4.0 wolfSSL API

wolfSSL_CTX_load_static_memory

Synopsis:

#include <wolfssl/ssl.h>

int wolfSSL_CTX_load_static_memory(WOLFSSL_CTX** ctx, wolfSSL_method_func method, unsigned char* buf, unsigned int sz, int flag, int max);

Description:

This function is used to set aside static memory for a CTX. Memory set aside is then used for the CTX’s lifetime and for any SSL objects created from the CTX. By passing in a NULL ctx pointer and a wolfSSL_method_func function the creation of the CTX itself will also use static memory. wolfSSL_method_func has the function signature of WOLFSSL_METHOD* (*wolfSSL_method_func)(void* heap);.
Passing in 0 for max makes it behave as if not set and no max concurrent use restrictions is in place.
The flag value passed in determines how the memory is used and behavior while operating. Available flags are the following.

0 - default general memory

WOLFMEM_IO_POOL - used for input/output buffer when sending receiving messages.
Overrides general memory, so all memory in buffer passed in is used for IO.
WOLFMEM_IO_FIXED - same as WOLFMEM_IO_POOL but each SSL now keeps two
buffers to themselves for their lifetime.
WOLFMEM_TRACK_STATS - each SSL keeps track of memory stats while running.

Return Values:

If successful, SSL_SUCCESS will be returned.

All unsuccessful return values will be less than 0 or equal to SSL_FAILURE.

Parameters:

ctx - address of pointer to a WOLFSSL_CTX structure.

method - function to create protocol. (should be NULL if ctx is not also NULL)

buf - memory to use for all operations.

sz - size of memory buffer being passed in.

flag - type of memory.

max- max concurrent operations.

Example:

WOLFSSL_CTX* ctx;

WOLFSSL* ssl;
int ret;
unsigned char memory[MAX];

int memorySz = MAX;

unsigned char IO[MAX];

int IOSz = MAX;
int flag = WOLFMEM_IO_FIXED | WOLFMEM_TRACK_STATS;

...

// create ctx also using static memory, start with general memory to use

ctx = NULL:

ret = wolfSSL_CTX_load_static_memory(&ctx, wolfSSLv23_server_method_ex, memory, memorySz, 0, MAX_CONCURRENT_HANDSHAKES);

if (ret != SSL_SUCCESS) {

// handle error case

}

// load in memory for use with IO

ret = wolfSSL_CTX_load_static_memory(&ctx, NULL, IO, IOSz, flag, MAX_CONCURRENT_IO);

if (ret != SSL_SUCCESS) {

// handle error case

}
...

See Also:

wolfSSL_CTX_new
wolfSSL_CTX_is_static_memory
wolfSSL_is_static_memory

wolfSSL_CTX_is_static_memory

Synopsis:

#include <wolfssl/ssl.h>

int wolfSSL_CTX_is_static_memory(WOLFSSL_CTX* ctx, WOLFSSL_MEM_STATS* mem_stats);

Description:

This function does not change any of the connections behavior and is used only for gathering information about the static memory usage.

Return Values:

A value of 1 is returned if using static memory for the CTX is true.

0 is returned if not using static memory.

Parameters:

ctx - a pointer to a WOLFSSL_CTX structure, created using wolfSSL_CTX_new().

mem_stats - structure to hold information about static memory usage.

Example:

WOLFSSL_CTX* ctx;
int ret;
WOLFSSL_MEM_STATS mem_stats;

...

//get information about static memory with CTX

ret = wolfSSL_CTX_is_static_memory(ctx, &mem_stats);

if (ret == 1) {

        // handle case of is using static memory
// print out or inspect elements of mem_stats

}

if (ret == 0) {

        //handle case of ctx not using static memory

}


See Also:

wolfSSL_CTX_new
wolfSSL_CTX_load_static_memory
wolfSSL_is_static_memory

wolfSSL_is_static_memory

Synopsis:

#include <wolfssl/ssl.h>

int wolfSSL_is_static_memory(WOLFSSL* ssl, WOLFSSL_MEM_CONN_STATS* mem_stats);

Description:

wolfSSL_is_static_memory is used to gather information about a SSL’s static memory usage. The return value indicates if static memory is being used and WOLFSSL_MEM_CONN_STATS will be filled out if and only if the flag WOLFMEM_TRACK_STATS was passed to the parent CTX when loading in static memory.

Return Values:

A value of 1 is returned if using static memory for the CTX is true.

0 is returned if not using static memory.

Parameters:

ssl - a pointer to a WOLFSSL structure, created using wolfSSL_new().

mem_stats - structure to contain static memory usage.

Example:

WOLFSSL* ssl;
int ret;
WOLFSSL_MEM_CONN_STATS mem_stats;

...

ret = wolfSSL_is_static_memory(ssl, mem_stats);

if (ret == 1) {

        // handle case when is static memory

        // investigate elements in mem_stats if WOLFMEM_TRACK_STATS flag

}

...

See Also:

wolfSSL_new

wolfSSL_CTX_is_static_memory

wolfSSLv3_server_method_ex

wolfSSLv3_client_method_ex

wolfTLSv1_server_method_ex

wolfTLSv1_client_method_ex

wolfTLSv1_1_server_method_ex
wolfTLSv1_1_client_method_ex

wolfTLSv1_2_server_method_ex

wolfTLSv1_2_client_method_ex

wolfSSLv23_server_method_ex

wolfSSLv23_client_method_ex

Synopsis:

#include <wolfssl/ssl.h>

WOLFSSL_METHOD* (*wolfSSL_method_func)(void* heap)

Description:

These functions have the same behavior as their counterparts (functions with not having _ex) except that they do not create WOLFSSL_METHOD* using dynamic memory. The functions will use the heap hint passed in to create a new WOLFSSL_METHOD struct.

Return Values:

A value of WOLFSSL_METHOD pointer if success.

NULL is returned in error cases.

Parameters:

heap - a pointer to a heap hint for creating WOLFSSL_METHOD struct.

Example:

WOLFSSL_CTX* ctx;
int ret;

...

ctx = NULL:

ret = wolfSSL_CTX_load_static_memory(&ctx, wolfSSLv23_server_method_ex, memory, memorySz, 0, MAX_CONCURRENT_HANDSHAKES);

if (ret != SSL_SUCCESS) {

// handle error case

}

...

See Also:

wolfSSL_new

wolfSSL_CTX_new
wolfSSL_CTX_free


4.1 wolfCrypt API

wolfSSL_MemoryPaddingSz

Synopsis:

#include <wolfssl/wolfcrypt/memory.h>

int wolfSSL_MemoryPaddingSz(void);

Description:

This function calculates the amount of padding needed with each bucket and is for informational purposes. The padding value is used for holding a memory management structure.

Return Values:

Returns padding size needed with each memory bucket.

Parameters:

None

Example:

...

printf(“with current setup %d padding is used\n”, wolfSSL_MemoryPaddingSz());

See Also:

wolfSSL_CTX_new
wolfSSL_CTX_is_static_memory
wolfSSL_is_static_memory

wc_LoadStaticMemory

wolfSSL_StaticBufferSz

Synopsis:

#include <wolfssl/wolfcrypt/memory.h>

int wolfSSL_StaticBufferSz(byte* buf, word32 sz, int flag);

Description:

This function calculates the optimum buffer size given the preset memory buckets. The value returned is rounded down.

Return Values:

Returns optimum buffer size to use for no trailing memory space unused in the buffer.

Parameters:

buf - address of the buffer, needed for calculating memory alignment requirements.

sz - size in bytes of the buffer

flag - type of memory desired

Example:


int ret;
unsigned char memory[MAX];

int memorySz = MAX;
int flag = WOLFMEM_IO_FIXED | WOLFMEM_TRACK_STATS;

...

ret = wolfSSL_StaticBufferSz(memory, memorySz, flag);

//trim memory buffer based on ret
...

See Also:

wolfSSL_CTX_new
wolfSSL_CTX_is_static_memory
wolfSSL_is_static_memory

wc_LoadStaticMemory

wc_LoadStaticMemory

Synopsis:

#include <wolfssl/wolfcrypt/memory.h>

int wc_LoadStaticMemory(WOLFSSL_HEAP_HINT* hint, unsigned char* buf, unsigned int sz, int flag, int max);

Description:

This function is used to set aside static memory for wolfCrypt use. Memory can be used by passing the created heap hint into functions. An example of this is when calling wc_InitRng_ex. The flag value passed in determines how the memory is used and behavior while operating, in general wolfCrypt operations will use memory from a WOLFMEM_GENERAL pool. Available flags are the following.

WOLFMEM_GENERAL - default general memory

WOLFMEM_IO_POOL - used for input/output buffer when sending receiving messages.
Overrides general memory, so all memory in buffer passed in is used for IO.
WOLFMEM_IO_FIXED - same as WOLFMEM_IO_POOL but each SSL now keeps two
buffers to themselves for their lifetime.
WOLFMEM_TRACK_STATS - each SSL keeps track of memory stats while running.

Return Values:

If successful, 0 will be returned.

All unsuccessful return values will be less than 0.

Parameters:

hint- WOLFSSL_HEAP_HINT structure to use

buf - memory to use for all operations.

sz - size of memory buffer being passed in.

flag - type of memory.

max- max concurrent operations (handshakes, IO). Currently not as applicable with wolfCrypt operations but retained for the instance of when used with TLS connections.

Example:

WOLFSSL_HEAP_HINT hint;
int ret;
unsigned char memory[MAX];

int memorySz = MAX;
int flag = WOLFMEM_GENERAL | WOLFMEM_TRACK_STATS;

...

// load in memory for use

ret = wc_LoadStaticMemory(&hint, memory, memorySz, flag, 0);

if (ret != SSL_SUCCESS) {

// handle error case

}

ret = wc_InitRng_ex(&rng, hint, 0);

// check ret value

See Also:

5.0 Tuning


To help in the case of limited resources and to reduce the overhead of using memory buckets, tuning can be done with setting the macros WOLFMEM_BUCKETS and WOLFMEM_DIST. For example configuring with the following reduced the peak usage of the cipher suite ECDHE-ECDSA-AES256-SHA384:


./configure --enable-staticmemory --enable-psk C_EXTRA_FLAGS="-DALT_ECC_SIZE -DWOLFSSL_DEBUG_MEMORY -DWOLFMEM_BUCKETS=64,256,384,432,512,1632,2976,3456,16128 -DWOLFMEM_DIST=16,8,6,4,6,3,2,1,1" --disable-extended-master --enable-debug

The way this would be interpreted is 1 bucket of 16128 bytes each pass, 1 of 3456 bytes, 2 of 2976 bytes and so on, matching the distribution number to the bucket size. WOLFMEM_BUCKETS sizes being a list with the smallest on the left and largest on the right in order.


In the case that memory alignment needs to be adjusted it can be set by using the macro WOLFSSL_STATIC_ALIGN. By default this is set to be 16 byte aligned but the following gives an example of setting to 8 byte aligned.

./configure --enable-staticmemory C_EXTRA_FLAGS=-DWOLFSSL_STATIC_ALIGN=8

5.1 Tuning IO Size
This accounts for if controlling both server and client side and using the --enable-maxfragment feature. When using a set TLS packet size the static IO buffer can be reduced with setting the macro WOLFMEM_IO_SZ. The default size for WOLFMEM_IO_SZ is 16,992 bytes to account for the largest packet size possibly seen when connecting to uncontrolled end points but if reducing the packet size in a controlled connection this size then becomes wasteful. An example of setting this with using autotools would be “./configure --enable-staticmemory C_EXTRA_FLAGS=-DWOLFMEM_IO_SZ=660”. For more information about setting the fragment size with TLS connections see documentation on wolfSSL_CTX_UseMaxFragment and wolfSSL_UseMaxFragment.