Feature/smp granular locks v4#1154
Conversation
|
@chinglee-iot @aggarg This PR introduces the granular locks changes to the FreeRTOS kernel. Please have a look and we could have discussions/changes in this PR context. Thank you. cc: @ESP-Marius @Dazza0 |
|
Thank you for your contribution, I'll forward this request to the team. There are few error in the PR can you please try fixing them |
|
Hello @sudeep-mohanty, I am just following up if you had time to fix the build issues |
|
@rawalexe Yes! I shall work on the failures and would also do some refactoring for an easier review process. For now, I've put this PR in draft. |
0159a4a to
5bf8c33
Compare
|
Hi @sudeep-mohanty, |
5bf8c33 to
9f8acc7
Compare
Hi @ActoryOu, I've made some updates which should fix the CI failures however I could not understand why the link-verifier action fails. Seems more like a script failure to me. So this action would still fail. If you have more information on what is causing it, could you let me know and I shall fix it. Thanks. |
c79962d to
f50d476
Compare
|
Created a PR to fix the CI failure - FreeRTOS/FreeRTOS#1292 |
70eb466 to
edc1c98
Compare
|
b20d191 to
1b91c54
Compare
prvLockQueue() and prvUnlockQueue() helper functions now only take the ISR locks because they are always called with the task lock already taken. However, these functions must also increment and decrement the critical nesting count as both locks get taken eventually. This commit fixes it.
…itical sections Stream buffer task lists (xTaskWaitingToSend and xTaskWaitingToReceive) are updated without critical sections. These objects are shared objects and can be updated from an ISR context as well. Hence, these lists must always be updated in critical sections.
5830f5c to
e850728
Compare
|
2555192 to
5463443
Compare
5463443 to
d33a460
Compare
|
|
There was a problem hiding this comment.
- Update to adapt to the unit test, C89 and GNU C extension.
- Suggest not to include configQUEUE_DIRECT_TRANSFER. We can consider this feature individually.
- Suggest to remove configLIGHTWEIGHT_CRITICAL_SECTION.
- Documentation for 'prvKernelEnterISROnlyCritical' and 'prvKernelExitISROnlyCritical'. The name of the function may be updated to
prvKernelEnterNestedCriticalSectionor other to imply this should be called when task lock of a data group is acquired already. - queue prvNotifyQueueSetContainer take only ISR spinlock only
- xTaskIncrementTick function update for vApplicationTickHook()
- Use macro to reduce lines of code for portBASE_TYPE_ENTER/EXIT_CRITICAL
| #if ( portUSING_GRANULAR_LOCKS == 1 ) | ||
| #define taskDATA_GROUP_UNLOCK( pxTaskSpinlock ) \ | ||
| ( { \ | ||
| portRELEASE_SPINLOCK( portGET_CORE_ID(), ( portSPINLOCK_TYPE * ) ( pxTaskSpinlock ) ); \ | ||
| /* Re-enable preemption after releasing the task spinlock. */ \ | ||
| xTaskPreemptionEnableWithYieldStatus( NULL ); \ | ||
| } ) | ||
| #endif /* #if ( portUSING_GRANULAR_LOCKS == 1 ) */ |
There was a problem hiding this comment.
Suggest not to use statement expression GNU C extension.
| #if ( ( portUSING_GRANULAR_LOCKS == 1 ) && ( configUSE_TICK_HOOK == 1 ) ) | ||
| { | ||
| xApplicationTickRequired = pdTRUE; | ||
| } | ||
| #else | ||
| { | ||
| vApplicationTickHook(); | ||
| } | ||
| #endif |
There was a problem hiding this comment.
| #if ( ( portUSING_GRANULAR_LOCKS == 1 ) && ( configUSE_TICK_HOOK == 1 ) ) | |
| { | |
| xApplicationTickRequired = pdTRUE; | |
| } | |
| #else | |
| { | |
| vApplicationTickHook(); | |
| } | |
| #endif | |
| #if ( configUSE_TICK_HOOK == 1 ) | |
| { | |
| #if ( portUSING_GRANULAR_LOCKS == 1 ) | |
| { | |
| xApplicationTickRequired = pdTRUE; | |
| } | |
| #else | |
| { | |
| vApplicationTickHook(); | |
| } | |
| #endif | |
| } | |
| #endif |
| traceENTER_xTaskRemoveFromEventList( pxEventList ); | ||
|
|
||
| BaseType_t xReturn; |
There was a problem hiding this comment.
Declaration before statement to comply with C89.
| traceENTER_xTaskRemoveFromEventList( pxEventList ); | |
| BaseType_t xReturn; | |
| BaseType_t xReturn; | |
| traceENTER_xTaskRemoveFromEventList( pxEventList ); | |
| traceENTER_xTaskRemoveFromEventListFromISR( pxEventList ); | ||
|
|
||
| #if ( configUSE_TICKLESS_IDLE != 0 ) | ||
| BaseType_t xReturn; |
There was a problem hiding this comment.
Declaration before statement to comply with C89.
| traceENTER_xTaskRemoveFromEventListFromISR( pxEventList ); | |
| #if ( configUSE_TICKLESS_IDLE != 0 ) | |
| BaseType_t xReturn; | |
| BaseType_t xReturn; | |
| traceENTER_xTaskRemoveFromEventListFromISR( pxEventList ); | |
| #if ( ( portUSING_GRANULAR_LOCKS == 1 ) && ( configNUMBER_OF_CORES > 1 ) ) | ||
| UBaseType_t uxSavedInterruptStatus; | ||
|
|
||
| /* Lock the kernel data group as we are about to access its members */ | ||
| uxSavedInterruptStatus = kernelENTER_CRITICAL_FROM_ISR(); | ||
| { | ||
| /* If a task is blocked on a kernel object then xNextTaskUnblockTime | ||
| * might be set to the blocked task's time out time. If the task is | ||
| * unblocked for a reason other than a timeout xNextTaskUnblockTime is | ||
| * normally left unchanged, because it is automatically reset to a new | ||
| * value when the tick count equals xNextTaskUnblockTime. However if | ||
| * tickless idling is used it might be more important to enter sleep mode | ||
| * at the earliest possible time - so reset xNextTaskUnblockTime here to | ||
| * ensure it is updated at the earliest possible time. */ | ||
| prvResetNextTaskUnblockTime(); | ||
| xReturn = prvTaskRemoveFromEventList( pxEventList ); | ||
| } | ||
| #endif | ||
| } | ||
| else | ||
| { | ||
| /* The delayed and ready lists cannot be accessed, so hold this task | ||
| * pending until the scheduler is resumed. */ | ||
| listINSERT_END( &( xPendingReadyList ), &( pxUnblockedTCB->xEventListItem ) ); | ||
| } | ||
| kernelEXIT_CRITICAL_FROM_ISR( uxSavedInterruptStatus ); | ||
| #else | ||
| xReturn = prvTaskRemoveFromEventList( pxEventList ); | ||
| #endif |
There was a problem hiding this comment.
Declare uxSavedInterruptStatus within brackets.
| #if ( ( portUSING_GRANULAR_LOCKS == 1 ) && ( configNUMBER_OF_CORES > 1 ) ) | |
| UBaseType_t uxSavedInterruptStatus; | |
| /* Lock the kernel data group as we are about to access its members */ | |
| uxSavedInterruptStatus = kernelENTER_CRITICAL_FROM_ISR(); | |
| { | |
| /* If a task is blocked on a kernel object then xNextTaskUnblockTime | |
| * might be set to the blocked task's time out time. If the task is | |
| * unblocked for a reason other than a timeout xNextTaskUnblockTime is | |
| * normally left unchanged, because it is automatically reset to a new | |
| * value when the tick count equals xNextTaskUnblockTime. However if | |
| * tickless idling is used it might be more important to enter sleep mode | |
| * at the earliest possible time - so reset xNextTaskUnblockTime here to | |
| * ensure it is updated at the earliest possible time. */ | |
| prvResetNextTaskUnblockTime(); | |
| xReturn = prvTaskRemoveFromEventList( pxEventList ); | |
| } | |
| #endif | |
| } | |
| else | |
| { | |
| /* The delayed and ready lists cannot be accessed, so hold this task | |
| * pending until the scheduler is resumed. */ | |
| listINSERT_END( &( xPendingReadyList ), &( pxUnblockedTCB->xEventListItem ) ); | |
| } | |
| kernelEXIT_CRITICAL_FROM_ISR( uxSavedInterruptStatus ); | |
| #else | |
| xReturn = prvTaskRemoveFromEventList( pxEventList ); | |
| #endif | |
| #if ( ( portUSING_GRANULAR_LOCKS == 1 ) && ( configNUMBER_OF_CORES > 1 ) ) | |
| { | |
| UBaseType_t uxSavedInterruptStatus; | |
| /* Lock the kernel data group as we are about to access its members */ | |
| uxSavedInterruptStatus = kernelENTER_CRITICAL_FROM_ISR(); | |
| { | |
| xReturn = prvTaskRemoveFromEventList( pxEventList ); | |
| } | |
| kernelEXIT_CRITICAL_FROM_ISR( uxSavedInterruptStatus ); | |
| } | |
| #else | |
| { | |
| xReturn = prvTaskRemoveFromEventList( pxEventList ); | |
| } | |
| #endif |
| kernelENTER_CRITICAL(); | ||
| #endif | ||
| { | ||
| const BaseType_t xCoreID = portGET_CORE_ID(); |
There was a problem hiding this comment.
Unused variable.
| const BaseType_t xCoreID = portGET_CORE_ID(); |
| * | ||
| * @return pdTRUE if the queue contains no items, otherwise pdFALSE. | ||
| */ | ||
| static BaseType_t prvIsQueueEmpty( const Queue_t * pxQueue ) PRIVILEGED_FUNCTION; |
There was a problem hiding this comment.
The queue spinlock state is changed in this function.
Remove const prefix both declaration and implementation.
| static BaseType_t prvIsQueueEmpty( Queue_t * pxQueue ) PRIVILEGED_FUNCTION; |
| * | ||
| * @return pdTRUE if there is no space, otherwise pdFALSE; | ||
| */ | ||
| static BaseType_t prvIsQueueFull( const Queue_t * pxQueue ) PRIVILEGED_FUNCTION; |
There was a problem hiding this comment.
The queue spinlock state is changed in this function.
Remove const prefix both declaration and implementation.
| portGET_SPINLOCK( xCoreID, ( portSPINLOCK_TYPE * ) &( pxQueue->pxQueueSetContainer->xTaskSpinlock ) ); | ||
| portGET_SPINLOCK( xCoreID, ( portSPINLOCK_TYPE * ) &( pxQueue->pxQueueSetContainer->xISRSpinlock ) ); | ||
| { | ||
| xReturn = prvNotifyQueueSetContainerGeneric( pxQueue, pdFALSE ); | ||
| } | ||
| portRELEASE_SPINLOCK( xCoreID, ( portSPINLOCK_TYPE * ) &( pxQueue->pxQueueSetContainer->xISRSpinlock ) ); | ||
| portRELEASE_SPINLOCK( xCoreID, ( portSPINLOCK_TYPE * ) &( pxQueue->pxQueueSetContainer->xTaskSpinlock ) ); |
There was a problem hiding this comment.
'prvNotifyQueueSetContainer' is called within queue critical section. Therefore, only ISR lock is required here.
| portGET_SPINLOCK( xCoreID, ( portSPINLOCK_TYPE * ) &( pxQueue->pxQueueSetContainer->xTaskSpinlock ) ); | |
| portGET_SPINLOCK( xCoreID, ( portSPINLOCK_TYPE * ) &( pxQueue->pxQueueSetContainer->xISRSpinlock ) ); | |
| { | |
| xReturn = prvNotifyQueueSetContainerGeneric( pxQueue, pdFALSE ); | |
| } | |
| portRELEASE_SPINLOCK( xCoreID, ( portSPINLOCK_TYPE * ) &( pxQueue->pxQueueSetContainer->xISRSpinlock ) ); | |
| portRELEASE_SPINLOCK( xCoreID, ( portSPINLOCK_TYPE * ) &( pxQueue->pxQueueSetContainer->xTaskSpinlock ) ); | |
| portGET_SPINLOCK( xCoreID, ( portSPINLOCK_TYPE * ) &( pxQueue->pxQueueSetContainer->xISRSpinlock ) ); | |
| { | |
| xReturn = prvNotifyQueueSetContainerGeneric( pxQueue, pdFALSE ); | |
| } | |
| portRELEASE_SPINLOCK( xCoreID, ( portSPINLOCK_TYPE * ) &( pxQueue->pxQueueSetContainer->xISRSpinlock ) ); |
| static void prvKernelEnterISROnlyCritical( void ); | ||
|
|
||
| static void prvKernelExitISROnlyCritical( void ); | ||
|
|
There was a problem hiding this comment.
Suggest to add internal function documentation here.
These two function are used when enter nested data group and kernel critical section.
| portBASE_TYPE_ENTER_CRITICAL(); | ||
| #if ( ( configNUMBER_OF_CORES > 1 ) ) | ||
| { | ||
| kernelENTER_CRITICAL(); |
There was a problem hiding this comment.
SMP FreeRTOS may not be able to benefit from portBASE_TYPE_ENTER/EXIT_CRITICAL due to multiple access from different core at the same time. We can use macro to reduce the line of code here and similar change below.
| @@ -4907,7 +5680,15 @@ BaseType_t xTaskIncrementTick( void ) | |||
| * count is being unwound (when the scheduler is being unlocked). */ | |||
| if( xPendedTicks == ( TickType_t ) 0 ) | |||
| { | |||
| vApplicationTickHook(); | |||
| #if ( ( portUSING_GRANULAR_LOCKS == 1 ) && ( configUSE_TICK_HOOK == 1 ) ) | |||
There was a problem hiding this comment.
| #if ( ( portUSING_GRANULAR_LOCKS == 1 ) && ( configUSE_TICK_HOOK == 1 ) ) | |
| #if ( portUSING_GRANULAR_LOCKS == 1 ) |
When TCB locks are used, we observe a race condition before taking a deferred action which can result in a task being orphaned and hence does not undergo the deferred action. THis can lead to memory leaks. This commit fixes the issue by ensuring that we do not allow a task to yield when a deferred action is pending.
…evelopment Integrate latest granular lock reconciliation changes from the development branch (chinglee-iot/FreeRTOS-Kernel, branch dev/update_for_granular_lock_demo_TCB_lock) to establish a common baseline for continued granular lock development. Key changes: - Eliminate configUSE_TCB_DATA_GROUP_LOCK; TCB locks now integral to portUSING_GRANULAR_LOCKS - Convert taskDATA_GROUP_ENTER_CRITICAL to function for run-state checking - Rename prvTaskPreemptionEnable to xTaskPreemptionEnableWithYieldStatus - Add prvTaskDataGroupCheckForRunStateChange, prvKernelEnterISROnlyCritical, xTaskUnlockCanYield - Restructure prvYieldCore and prvYieldForTask with TCB spinlock protection - Add scheduler suspension in event group list walking - Add queue set container spinlock protection - Enable single-priority mode with preemption disable - Expose timer spinlocks for testing Bug fixes applied during reconciliation: - Fix typo: vTaskTCBExtiCritical -> vTaskTCBExitCritical - Fix unused variable in xTaskPreemptionEnableWithYieldStatus - Fix broken tick hook preprocessor guard - Fix unused variable in prvNotifyQueueSetContainerFromISR
Under granular locks, queue.c paths that unblock a waiting task performed an outer listLIST_IS_EMPTY() check on the event list before calling xTaskRemoveFromEventList[FromISR](). The outer check ran with only the queue lock held, while the event list is protected by the kernel ISR lock. Between the outer check (non-empty) and the inner call, the tick ISR on another core could remove the waiting task via timeout, leaving the event list empty. The unblock was then silently skipped while the caller still owned the data, violating the priority contract under configRUN_MULTIPLE_PRIORITIES=0. prvTaskRemoveFromEventList() already re-checks the list under the kernel ISR lock (tasks.c) and safely returns pdFALSE on an empty list. The outer check is therefore both redundant and racy. This commit removes the outer listLIST_IS_EMPTY() guard at every race-prone call site in queue.c (10 sites covering xQueueGenericReset, xQueueGenericSend, xQueueGenericSendFromISR, xQueueGiveFromISR, xQueueSemaphoreTake, xQueuePeek, and prvNotifyQueueSetContainer). Where the original code branched on listLIST_IS_EMPTY to fall through to an else-if (xYieldRequired) path, that branch is preserved by chaining off the xTaskRemoveFromEventList() return value, which yields pdFALSE in the empty-list case. Sites in prvUnlockQueue retain their outer empty check because that check serves as the drain-loop termination via break, and the surrounding queue ISR spinlock prevents concurrent additions to the wait list there.
…P_UNLOCK The taskDATA_GROUP_UNLOCK macro relied on the GNU C statement-expression extension to return the yield status of the underlying unlock operation, which is not portable to strict-C compilers. Rewrite the macro using standard C constructs while preserving its return-value semantics for existing callers.
…ontainer prvNotifyQueueSetContainer is invoked from within the source queue's critical section, which already holds the data-group's task spinlock and has interrupts disabled. Acquiring the queue-set container's task spinlock in addition to its ISR spinlock was therefore redundant. Take only the ISR spinlock, matching the lock hierarchy of the ISR variant.
Replace six identical #if ( configNUMBER_OF_CORES > 1 ) ... #else ... #endif blocks (in uxTaskPriorityGet and two sibling APIs) with a single pair of helper macros that resolve to kernelENTER/EXIT_CRITICAL on SMP and the lighter portBASE_TYPE_ENTER/EXIT_CRITICAL on single-core. Removes ~30 lines of repeated preprocessor scaffolding without changing the generated code.
* xTaskRemoveFromEventList and xTaskRemoveFromEventListFromISR now declare the BaseType_t xReturn local before the traceENTER_* invocation so the function bodies remain valid C89 (declarations must precede statements at block scope). For the FromISR variant, uxSavedInterruptStatus is also moved inside the ( portUSING_GRANULAR_LOCKS == 1 ) && ( configNUMBER_OF_CORES > 1 ) block so it is not declared in scopes that never reference it. * In xTaskIncrementTick the two inner #if guards selecting between setting xApplicationTickRequired and calling vApplicationTickHook now check only portUSING_GRANULAR_LOCKS, and the second site is wrapped in #if ( configUSE_TICK_HOOK == 1 ) so the structure of the two sites is symmetric. * xTaskPreemptionEnableWithYieldStatus drops an unused const BaseType_t xCoreID local that was never referenced after the surrounding rework. * The forward declarations of prvKernelEnterISROnlyCritical and prvKernelExitISROnlyCritical gain a block comment describing how they extend a surrounding data-group critical section into the kernel data group without disturbing the interrupt state managed by the outer section.
64219a4 to
f06f3a5
Compare
configLIGHTWEIGHT_CRITICAL_SECTION enabled an alternative critical section implementation that takes only the kernel ISR spinlock (without the task spinlock) for preemption-disable / preemption-enable paths. The feature is out of scope for the upstream granular-locks PR. Remove every #if ( configLIGHTWEIGHT_CRITICAL_SECTION == 1 ) block from tasks.c (forward declaration and definitions of prvLightWeightCheckForRunStateChange, vKernelLightWeightEnterCritical, vKernelLightWeightExitCritical), the matching public declarations from include/freertos/task.h, the #ifndef default definition from include/freertos/FreeRTOS.h, and the #define configLIGHTWEIGHT_CRITICAL_SECTION line from FreeRTOSConfig.h. The removed functions had no external callers.
|



This PR adds support for granular locking to the FreeRTOS kernel.
Description
Granular locking introduces the concept of having localized locks per kernel data group for SMP configuration. This method is an optional replacement of the existing kernel locks and is controlled by a new port layer configuration, viz.,
portUSING_GRANULAR_LOCKS. More details about the approach can be found here.Test Steps
The implementation has been tested on Espressif SoC targets, viz., the ESP32 using the ESP-IDF framework.
1. Testing on an
esp32targetesp32, setup the ESP-IDF environment on your local machine. The steps to follow are listed in the Getting Started Guide.components/freertos/test_apps/freertoswhere all the test cases are located.performancesubfolder at the same location.idf.py seet-target esp32.menuconfigoptions. To do this you must enter the command-idf.py menuconfig->Component config->FreeRTOS->Kernel->Run the Amazon SMP FreeRTOS kernel instead (FEATURE UNDER DEVELOPMENT). Save the configuration and exit the menuconfig.idf.py build flash monitor.TODO
Checklist:
Related Issue
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.