[Rtai] Source of big latencies???

Federico Bergero fedebergero at gmail.com
Fri Sep 30 21:08:23 CEST 2011


Sorry, now that I see it, it is not a bug but it is confusing that if
you set CONFIG_RTAI_HW_SMI_ALL the other flags don't have meaning. In
the normal case it is OK but in my case (where GBL_SMI_EN_BIT cannot
be written) masking the other
bits(INTEL_USB2_EN_BIT,LEGACY_USB2_EN_BIT,etc) did nothing because I
had CONFIG_RTAI_HW_SMI_ALL also set.

Another nice to have. The module should check after masking the bits
if they were actually masked (in my case they didn't change) and
should alert the user if it couldn't do it by a proper message.

Thanks again!

Fede
2011/9/30 Federico Bergero <fedebergero at gmail.com>:
> Hi.
> I found a solution.
> The SMI module  had a bug  (rtai-3.8.1) which, I see is corrected in
> magma, where
>
> unsigned long hal_smi_masked_bits = 0
> #if CONFIG_RTAI_HW_SMI_ALL
>    | GBL_SMI_EN_BIT
> #else
>
> the #else should be a #endif.
>
> Another thing I found is that the GBL_SMI_EN_BIT NEVER gets changed
> (even if the module writes a 0 to it). I assume this should be because
> the SMI_LOCK is enabled (didn't check it).
> Following the suggestion of Stephan Brauss, I disabled all others bits
> in the SMI_EN and now it works great.
>
> Perhaps the smi_module should, first disable SMI_LOCK, then disable
> SMI and then reenable SMI_LOCK (in the chance than a reset re-enables
> SMI).
>
> Thanks to everybody!
> Fede
>
> 2011/9/30 Giampaolo Bellini <iw2lsi at gmail.com>:
>> Hello Federico...
>> sorry... don't know why but I was assuming you where Italian... and as my
>> english is poor I decided to write you in my native language.
>> over all performances of the IEI board where good, but with sporadic (after
>> 1h or more) latencies with values about 500us or 1ms
>> I was no able to fix the problem, either with standard setsmi and/or with a
>> patched one I fix by myself to set the SMI-LOCK flag
>> unfortunatly it's not the first time I fall into big-latencies... last time
>> was with an ICOP board that has some strange behaviour due to it's fail-safe
>> redundancy policy that occurs exactly every hour.
>> if I where you, as first stage I'll check on google for the SMI-LOCK flag an
>> at RTAI setsmi sources in order to see if it take care about it.
>>
>>   hope you'll fix it
>>          Best Regards
>>                    Giampaolo
>>
>>
>> 2011/9/30 Federico Bergero <fedebergero at gmail.com>
>>>
>>> Ciao Giampaolo,
>>> (il mio italiano non è il migliore).
>>> Questo problema ti succedia periodicamente? A me mi succede ogni 2.43
>>> secondi.
>>> Tu non hai potuto risolverlo? Hai cambiato la scheda?
>>> Non so si io posso cambiare la scheda. È una EVGA X58 SLI LE
>>> (http://www.evga.com/articles/00438/)
>>>
>>> Traterò di vedere lo del SMI_LOCK.
>>>
>>> Grazie tante!
>>>
>>> Federico
>>>
>>> 2011/9/29 Giampaolo Bellini <iw2lsi at gmail.com>:
>>> > Ciao Federico
>>> >   tempo fa ho lavorato con una scheda della IEI (NOVA-9452) che montava
>>> > un
>>> > chipset ICH7M e riscontrai anche io dei problemi di latenza che non
>>> > riuscii
>>> > a risolvere in nessun modo su quella motherboard (provai circa una
>>> > trentina
>>> > di configurazioni diverse, che coinvolgevano anche la scheda di rete).
>>> > non so se il modulo che setta lo SMI in RTAI attivi anche il flag
>>> > SMI_LOCK
>>> > (che evita che un reset PCI riattivi lo SMI)... ma potresti dargli
>>> > un'occhiata...
>>> >  saluti
>>> >          Giampaolo
>>> >
>>> >
>>> > 2011/9/28 Federico Bergero <fedebergero at gmail.com>
>>> >>
>>> >> I recompiled the kernel without SMP so Linux is using ONE core.
>>> >> I still got those results.
>>> >> I will continue to investigate, but without any clues where to start.
>>> >> Is it posible that the smi modules is not turning off SMI? Can I tell
>>> >> if SMI is actually disabled?
>>> >>
>>> >> What else could be so periodic?
>>> >>
>>> >> Thanks
>>> >> Fede
>>> >>
>>> >> 2011/9/28 Paolo Mantegazza <mantegazza at aero.polimi.it>:
>>> >> > Federico Bergero wrote:
>>> >> >>
>>> >> >> The 82801JI (ICH10 Family) chipset is not listed in smi-module.c
>>> >> >> (I'm
>>> >> >> using rtai 3.8.1).
>>> >> >> I see that in the CVS is listed so now I'm going to try to compile
>>> >> >> magma from cvs.
>>> >> >> But anyway, I had added
>>> >> >> #ifndef PCI_DEVICE_ID_INTEL_82801JB_10
>>> >> >> #define PCI_DEVICE_ID_INTEL_82801JB_10 0x3a16
>>> >> >> #endif
>>> >> >> To the smi-module.c and I could installed it.
>>> >> >> The module told me
>>> >> >>
>>> >> >> [  196.245964] RTAI: Intel chipset found, enabling SMI workaround.
>>> >> >> [  196.245977] SMI configuration has been cleared, mask used = 1
>>> >> >> (saved mask setting 1).
>>> >> >
>>> >> > with that mask you have disable SMI completely, so the problem
>>> >> > shopuld
>>> >> > be
>>> >> > elswhere. Try to work UP first.
>>> >> >
>>> >> > paolo
>>> >> >
>>> >> >> But still got those big latencies...
>>> >> >>
>>> >> >> Do I have to run setsmi now? With what parameters?
>>> >> >>
>>> >> >>
>>> >> >> Thanks
>>> >> >> Fede
>>> >> >> 2011/9/28  <mantegazza at aero.polimi.it>:
>>> >> >>>>
>>> >> >>>> If it is of any help here's my lspci/dmesg.
>>> >> >>>>
>>> >> >>>> I see there a
>>> >> >>>> SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
>>> >> >>>>
>>> >> >>>> But I still get the Intel chipset not found
>>> >> >>>>
>>> >> >>>> Any ideas?
>>> >> >>>>
>>> >> >>> RTAI should find ICH10_1, anything greater is not set for a search
>>> >> >>> in
>>> >> >>> smi_module.c
>>> >> >>>
>>> >> >>> paolo
>>> >> >>>
>>> >> >>>> 2011/9/27 Federico Bergero <fedebergero at gmail.com>:
>>> >> >>>>>
>>> >> >>>>> Sorry, I didn't follow you Paolo.
>>> >> >>>>> I should disable ACPI from the BIOS or recompile the kernel
>>> >> >>>>> without
>>> >> >>>>> ACPI support?
>>> >> >>>>> I didn't understand neither what you said about SMI.
>>> >> >>>>> When I install the module I get:
>>> >> >>>>> insmod: error inserting './rtai_smi.ko': -1 No such device
>>> >> >>>>> RTAI: Intel chipset not found.
>>> >> >>>>>
>>> >> >>>>> How can I know 1) if I have a intel chipset 2) If I don't have
>>> >> >>>>> one,
>>> >> >>>>> I
>>> >> >>>>> need to do something else?
>>> >> >>>>>
>>> >> >>>>> Thanks in advance!
>>> >> >>>>> Fede
>>> >> >>>>>
>>> >> >>>>> 2011/9/27 Paolo Mantegazza <mantegazza at aero.polimi.it>:
>>> >> >>>>>>
>>> >> >>>>>> Federico Bergero wrote:
>>> >> >>>>>>>
>>> >> >>>>>>> Ok. I'll try setting only one core without HT in the bios.
>>> >> >>>>>>> That way Linux will only use one core.
>>> >> >>>>>>>
>>> >> >>>>>> I think there is no such a need, you have just to disable SMO on
>>> >> >>>>>> your
>>> >> >>>>>> system. A few times I've seen that just diabling ACPI set the PC
>>> >> >>>>>> UP.
>>> >> >>>>>> If you are using an INTEL chipset than the missing of it is
>>> >> >>>>>> likely
>>> >> >>>>>> related
>>> >> >>>>>> to the fact that SMI disabling RTAI need to have it listed.
>>> >> >>>>>>
>>> >> >>>>>> paolo
>>> >> >>>>>>
>>> >> >>>>>>> Anyway I'm using LXRT so I'm not using the UP nor the SMP (nor
>>> >> >>>>>>> the
>>> >> >>>>>>> MUP) scheduler,right?.
>>> >> >>>>>>>
>>> >> >>>>>>> I tried taking care of SMI but when I insert the module it
>>> >> >>>>>>> tells
>>> >> >>>>>>> me
>>> >> >>>>>>> "No intel chip detected", so I thought that didn't need to do
>>> >> >>>>>>> anything. Is this correct? Or I have to do something anyway?
>>> >> >>>>>>>
>>> >> >>>>>>> I'll reply later with the new results
>>> >> >>>>>>>
>>> >> >>>>>>> Thanks
>>> >> >>>>>>>
>>> >> >>>>>>> Fede
>>> >> >>>>>>>
>>> >> >>>>>>> 2011/9/27 Paolo Mantegazza <mantegazza at aero.polimi.it>:
>>> >> >>>>>>>>
>>> >> >>>>>>>> Federico Bergero wrote:
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> What do you mean by uni-processor?
>>> >> >>>>>>>>> I have disabled HT but still got 6 cores (a small SMP
>>> >> >>>>>>>>> system).
>>> >> >>>>>>>>> Anyway the test fails with only one thread, so I don't think
>>> >> >>>>>>>>> that
>>> >> >>>>>>>>> HT
>>> >> >>>>>>>>> or multiple cores have anything to do with it.
>>> >> >>>>>>>>> In theory RTAI shouldn't give the processor back to Linux if
>>> >> >>>>>>>>> the
>>> >> >>>>>>>>> RTAI
>>> >> >>>>>>>>> task is doing something (like in a busy wait), right? If so,
>>> >> >>>>>>>>> what
>>> >> >>>>>>>>> could have HIGHER priority than RTAI task? Bios? ACPI/APM?
>>> >> >>>>>>>>>
>>> >> >>>>>>>> To play with the config you should first configure Linux as
>>> >> >>>>>>>> UP.
>>> >> >>>>>>>> If
>>> >> >>>>>>>> the
>>> >> >>>>>>>> latency remains than you have to know what hardware you
>>> >> >>>>>>>> bought.
>>> >> >>>>>>>> If
>>> >> >>>>>>>> it
>>> >> >>>>>>>> does
>>> >> >>>>>>>> not than you have a clue related to what your hardware does
>>> >> >>>>>>>> SMP.
>>> >> >>>>>>>> Have
>>> >> >>>>>>>> cared
>>> >> >>>>>>>> of SMI already?
>>> >> >>>>>>>> Please keep posting to the list.
>>> >> >>>>>>>>
>>> >> >>>>>>>>
>>> >> >>>>>>>> paolo
>>> >> >>>>>>>>
>>> >> >>>>>>>>> Grazie tante!
>>> >> >>>>>>>>> Fede
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> 2011/9/27 Paolo Mantegazza <mantegazza at aero.polimi.it>:
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> Federico Bergero wrote:
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> I have disabled HT in the BIOS (now running with 6cores)
>>> >> >>>>>>>>>>> I'm still getting a source of high latencies.
>>> >> >>>>>>>>>>> I found out that these big latencies are PERIODIC (?)
>>> >> >>>>>>>>>>> They occur every 2.43 seg.
>>> >> >>>>>>>>>>> I'm attaching the test I have run.
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> I'm using the kernel from
>>> >> >>>>>>>>>>> http://hart.sourceforge.net/rtai_installation_10_04.html
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> Any ideas what could be going on?
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>> Have you tried the latency test after making your machine as
>>> >> >>>>>>>>>> if
>>> >> >>>>>>>>>> it
>>> >> >>>>>>>>>> was
>>> >> >>>>>>>>>> UP.
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> paolo
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>>> These is the output of the test:
>>> >> >>>>>>>>>>> [   58.336000] Error diff 2343675702
>>> >> >>>>>>>>>>> [   58.336000] Error diff 2343677010
>>> >> >>>>>>>>>>> [   59.628000] Error diff 2343704223
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> 2011/9/26 Paolo Mantegazza <mantegazza at aero.polimi.it>:
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> Federico Bergero wrote:
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>> Hi I'm working on a project with rtai.
>>> >> >>>>>>>>>>>>> I'm using a Intel i7 with 6 cores and HT so Linux sees 12
>>> >> >>>>>>>>>>>>> cores.
>>> >> >>>>>>>>>>>>> I'm running a real time LXRT-RTAI thread on each core
>>> >> >>>>>>>>>>>>> (indicating
>>> >> >>>>>>>>>>>>> RTAI
>>> >> >>>>>>>>>>>>> to do so with the cpus_allowed mask).
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>> Each thread runs a simulation of a system syncrhonized
>>> >> >>>>>>>>>>>>> with
>>> >> >>>>>>>>>>>>> real
>>> >> >>>>>>>>>>>>> time,
>>> >> >>>>>>>>>>>>> and I'm using rt_sleep/rt_busy_sleep to do the waits.
>>> >> >>>>>>>>>>>>> When running the simulation SOMETIMES I get a max
>>> >> >>>>>>>>>>>>> latencies
>>> >> >>>>>>>>>>>>> of
>>> >> >>>>>>>>>>>>> 900us
>>> >> >>>>>>>>>>>>> on some processor and 2us on others processors.
>>> >> >>>>>>>>>>>>> These BIG latencies occur randomly. Sometimes everything
>>> >> >>>>>>>>>>>>> runs
>>> >> >>>>>>>>>>>>> smoothly.
>>> >> >>>>>>>>>>>>> What could be the source of these lantencies? I have
>>> >> >>>>>>>>>>>>> turned
>>> >> >>>>>>>>>>>>> off
>>> >> >>>>>>>>>>>>> Freq
>>> >> >>>>>>>>>>>>> Scaling in Linux and SpeedStep in the BIOS.
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>> Also I don't quiet understand how the
>>> >> >>>>>>>>>>>>> rt_get_cpu_time_ns()
>>> >> >>>>>>>>>>>>> is
>>> >> >>>>>>>>>>>>> implemented on a SMP system, could someone explain me
>>> >> >>>>>>>>>>>>> this?
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>> Thanks Fede
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>> First of all do not use HT.
>>> >> >>>>>>>>>>>> On your machine rt_get_cpu_time_ns() simply reads and
>>> >> >>>>>>>>>>>> converts
>>> >> >>>>>>>>>>>> the
>>> >> >>>>>>>>>>>> APIC
>>> >> >>>>>>>>>>>> timer on each CPU. You can check if their are big
>>> >> >>>>>>>>>>>> difference
>>> >> >>>>>>>>>>>> intra
>>> >> >>>>>>>>>>>> CPUs
>>> >> >>>>>>>>>>>> by
>>> >> >>>>>>>>>>>> configuring RTAI to do so.
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> paolo
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> paolo
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>> --
>>> >> >>>>> I find television very educational. Every time someone switches
>>> >> >>>>> it
>>> >> >>>>> on,
>>> >> >>>>> I go into another room and read a good book.
>>> >> >>>>>
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> --
>>> >> >>>> I find television very educational. Every time someone switches it
>>> >> >>>> on,
>>> >> >>>> I go into another room and read a good book.
>>> >> >>>>
>>> >> >>>
>>> >> >>>
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> I find television very educational. Every time someone switches it on,
>>> >> I go into another room and read a good book.
>>> >> _______________________________________________
>>> >> Rtai mailing list
>>> >> Rtai at rtai.org
>>> >> https://mail.rtai.org/cgi-bin/mailman/listinfo/rtai
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> I find television very educational. Every time someone switches it on,
>>> I go into another room and read a good book.
>>
>>
>
>
>
> --
> I find television very educational. Every time someone switches it on,
> I go into another room and read a good book.
>



-- 
I find television very educational. Every time someone switches it on,
I go into another room and read a good book.


More information about the Rtai mailing list