POSIX - Switch from posix timers to a timer thread to fix signal handling with non-FreeRTOS pthreads

Improve upon the elegant approach of using signals to cause task/pthreads
suspension and scheduler execution by using directed signals.

This fixes:
- Deadlocks in non-FreeRTOS pthreads
- Multiple FreeRTOS tasks(pthreads) incorrectly running at the same time

By directing the signals using pthread_kill() the signal handler in the presently running
FreeRTOS task/pthread will be called, ensuring that the scheduler runs both in the context
of a FreeRTOS task/pthread and from the presently executing FreeRTOS task/pthread.

Details
==============

The POSIX port uses signals to preempt FreeRTOS tasks (implemented as pthreads), a very neat and elegant
approach to forcing tasks/pthreads to suspend and run the scheduler.

Signal handlers are process global.

Posix timers generate signals when the timer expires, and the signal is sent to the currently
running pthread.

In systems where there are pthreads that are NOT a result of creating FreeRTOS tasks, such as the
entry point thread that calls main(), or user created pthreads, this poses a serious issue.

While the POSIX port only allows a single FreeRTOS pthread to run at once, by causing all suspended
threads to not be scheduled due to their waiting on a pthread condition variable,
this isn't the case with non-FreeRTOS pthreads.

Thus it is possible that a non-FreeRTOS pthread is running when the timer expires and the signal
is generated. This results in the signal handler running in the non-FreeRTOS thread.

The sequence of events results in these events from signal handler context:
- vPortSystemTickHandler() being called
- The scheduler running
- Selecting another FreeRTOS task to run and switching the active task
- The newly selected task released from suspension by pthread_cond_signal()
- The presently active thread calling event_wait()
- The pthread calling pthread_cond_wait(), suspending the thread and allowing the host OS scheduler
  to schedule another thread to run.

If this occurs from a non-FreeRTOS thread this results in:
- The active FreeRTOS pthread (Task A/Thread A) continuing to run (as the signal handler that calls
  event_wait() ran instead in a non-FreeRTOS pthread.
- The pthread where the signal handler did run (Thread B) will call event_wait() and pthread_cond_wait(),
  but on the condition variable of the previously active FreeRTOS task, oops. This causes the
  non-FreeRTOS pthread to block unexpectedly relative to what the developer might have expected.
- The newly selected FreeRTOS Task (Task C/Thread C) will resume and start running.

At this point Task A/Thread A is running concurrently with Task C/Thread C. While this may not
necessarily be an issue, it does not replicate the expected behavior of a single Task running at
once.

Note that Thread B will resume if/when Task A/ThreadA is switched to. However, this could be delayed
by an arbitrary amount of time, or could never occur.

Also note that if there are multiple non-FreeRTOS pthreads that Thread D, E, F...etc could suffer the
same fate as Thread B, if the scheduler were to suspend Task C/Thread C and resume Task E/Thread E.

Implementation
==============

Timer details
-------------
A standalone pthread for the signal generation thread was chosen, rather than using
a posix timer_settime() handler function because the latter creates a temporary
pthread for each handler callback. This makes debugging much more difficult due to
gdb detecting the creation and destruction of these temporary threads.

Signal delivery
--------------
While signal handlers are per-thread, it is possible for pthreads to selectively block
signals, rather than using thread directed signals. However, the approach of blocking
signals in non-FreeRTOS pthreads adds complexity to each of these non-FreeRTOS pthreads
including ensuring that these signals are blocked at thread creation, prior to the thread
starting up. Directed signals removes the requirement for non-FreeRTOS pthreads to be aware
of and take action to protect against these signals, reducing complexity.

1 files changed, 27 insertions, 42 deletions

diff --git a/portable/ThirdParty/GCC/Posix/port.c b/portable/ThirdParty/GCC/Posix/port.c
index 5dc3d73e2..d312aa48d 100644
--- a/portable/ThirdParty/GCC/Posix/port.c
+++ b/portable/ThirdParty/GCC/Posix/port.c
@@ -60,6 +60,7 @@
 #include <sys/time.h>
 #include <sys/times.h>
 #include <time.h>
+#include <unistd.h>
 
 #ifdef __APPLE__
     #include <mach/mach_vm.h>
@@ -103,6 +104,9 @@ static pthread_t hMainThread = ( pthread_t ) NULL;
 static volatile BaseType_t uxCriticalNesting;
 /*-----------------------------------------------------------*/
 
+static pthread_t hTimerTickThread;
+static bool xTimerTickThreadShouldRun;
+
 static BaseType_t xSchedulerEnd = pdFALSE;
 /*-----------------------------------------------------------*/
 
@@ -231,6 +235,10 @@ BaseType_t xPortStartScheduler( void )
         sigwait( &xSignals, &iSignal );
     }
 
+    /* asking timer thread to shut down */
+    xTimerTickThreadShouldRun = false;
+    pthread_join( hTimerTickThread, NULL );
+
     /* Cancel the Idle task and free its resources */
     #if ( INCLUDE_xTaskGetIdleTaskHandle == 1 )
         vPortCancelThread( xTaskGetIdleTaskHandle() );
@@ -250,24 +258,8 @@ BaseType_t xPortStartScheduler( void )
 
 void vPortEndScheduler( void )
 {
-    struct itimerval itimer;
-    struct sigaction sigtick;
     Thread_t * xCurrentThread;
 
-    /* Stop the timer and ignore any pending SIGALRMs that would end
-     * up running on the main thread when it is resumed. */
-    itimer.it_value.tv_sec = 0;
-    itimer.it_value.tv_usec = 0;
-
-    itimer.it_interval.tv_sec = 0;
-    itimer.it_interval.tv_usec = 0;
-    ( void ) setitimer( ITIMER_REAL, &itimer, NULL );
-
-    sigtick.sa_flags = 0;
-    sigtick.sa_handler = SIG_IGN;
-    sigemptyset( &sigtick.sa_mask );
-    sigaction( SIGALRM, &sigtick, NULL );
-
     /* Signal the scheduler to exit its loop. */
     xSchedulerEnd = pdTRUE;
     ( void ) pthread_kill( hMainThread, SIG_RESUME );
@@ -366,38 +358,31 @@ static uint64_t prvStartTimeNs;
  * to adjust timing according to full demo requirements */
 /* static uint64_t prvTickCount; */
 
+static void* prvTimerTickHandler(void *arg)
+{
+    while( xTimerTickThreadShouldRun )
+    {
+        /*
+         * signal to the active task to cause tick handling or
+         * preemption (if enabled)
+         */
+        Thread_t * thread = prvGetThreadFromTask( xTaskGetCurrentTaskHandle() );
+        pthread_kill( thread->pthread, SIGALRM );
+
+        usleep( portTICK_RATE_MICROSECONDS );
+    }
+
+    return NULL;
+}
+
 /*
  * Setup the systick timer to generate the tick interrupts at the required
  * frequency.
  */
 void prvSetupTimerInterrupt( void )
 {
-    struct itimerval itimer;
-    int iRet;
-
-    /* Initialise the structure with the current timer information. */
-    iRet = getitimer( ITIMER_REAL, &itimer );
-
-    if( iRet == -1 )
-    {
-        prvFatalError( "getitimer", errno );
-    }
-
-    /* Set the interval between timer events. */
-    itimer.it_interval.tv_sec = 0;
-    itimer.it_interval.tv_usec = portTICK_RATE_MICROSECONDS;
-
-    /* Set the current count-down. */
-    itimer.it_value.tv_sec = 0;
-    itimer.it_value.tv_usec = portTICK_RATE_MICROSECONDS;
-
-    /* Set-up the timer interrupt. */
-    iRet = setitimer( ITIMER_REAL, &itimer, NULL );
-
-    if( iRet == -1 )
-    {
-        prvFatalError( "setitimer", errno );
-    }
+    xTimerTickThreadShouldRun = true;
+    pthread_create( &hTimerTickThread, NULL, prvTimerTickHandler, NULL );
 
     prvStartTimeNs = prvGetTimeNs();
 }