[Rtai] Problems with the LXRT real-time performance

[Christian Poessinger]

Hallo,

I'm currently writing my thesis with the subject: "Analysing the real-time
characteristics of RTOS"

One of my systems is a RTAI 3.6 based Pentium 4 machine with Debian Etch
and a 2.6.23 kernel. I measure the time from activating an Interrupt on
IRQ7 till a task is rescheduled by the ISR. I name it interrupt +
scheduling latency.

I've got two measure programs, one that is directly programmed for the
RTAI API and one that runs as a LXRT hard-real-time task in userspace with
a custom parallel port module for the interrupt stuff. When performing
measurements directly on the RTAI kernel, everything works really great
and I achive the worstcase latencys at 37 microseconds with one in a
million, when runnning a stresstest on my linux system with "stress --vm 2
--vm-bytes 128M".

Now when repeating the measurement with the LXRT task I see 999979 of the
measured values at 67 microseconds. The problem is that I also do find 20
values at a time of 311 microseconds and at worst one value at 463
microseconds, with the same stress command as mentioned above.

This behavior is fully reproducible. I wanted to examine my system with
LTT but I wasn't able to find any LTT patch for the 2.6.23 kernel nor any
option in the menuconfig for activating it. This is needed by the RTAI
system when I enable LTT in RTAI. LTTng didn't apply, too.

The funny thing is, when i decuple the number of measurements the count of
the outlier values are not decupled. If the worst comes to the worst they
double.

Currently tested options to avoid outliners:

 - locked application in ram mit mlockall()
 - disabled kernel support for paging of anonymous memory (swap)
 - disabled TSC by booting with notsc kernel command line option
 - deactivated DMA in the Kernel and on the harddrive
 - disable the CPU cache with a kernel module found at
http://www.rtlinux-gpl.org/mailing_list/rtl.w5archive/0103/msg00133.html
but then the system freezes
 - disabled the tickless kernel option
 - disabled HPET (High Precision Event Timer) and HRT (High Resolution Timer)
 - loaded the rtai_smi module to disable SMI

The outliners appear when I use the system in idle mode, as well as when
doing a "stress --vm 2 --vm-bytes 128M" or when using a custom memcpy()
while loop, to copy 128mb.





Here is the main() part with the realtime stuff of the LXRT measurement
sourcecode:



int main(void) {

    /* START REALTIME STUFF */
    RT_TASK *pstMainBuddy;
    int priority=0;   // Highest
    int stack_size=0; // Use default (512)
    int msg_size=0;   // Use default (256)

    /* Create the buddy of the main task */
    pstMainBuddy = rt_task_init( nam2num("getTT"), priority, stack_size,
msg_size);
    if (pstMainBuddy == NULL) {
        fprintf(stderr, "getTimeTest: Error creating the main buddy!\n");
        exit(1);
    }

    /* Disable Paging */
    mlockall(MCL_CURRENT | MCL_FUTURE);

    rt_make_hard_real_time();
    /* END REALTIME STUFF */

	    [...] // some non realtime stuff any mainly the measure functions

    munlockall();

    rt_make_soft_real_time();
    rt_task_delete(pstMainBuddy);

    return 0;
}


Best regards,
Christian Poessinger