__rdtscp calibration unstable under Linux on Intel Xeon X5550

I’m trying to use __rdtscp intrinsinc function to measure time intervals. Target platform is Linux x64, CPU Intel Xeon X5550. Although constant_tsc flag is set for this processor, calibrating __rdtscp gives very different results:

$ taskset -c 1 ./ticks
Ticks per usec: 256
$ taskset -c 1 ./ticks
Ticks per usec: 330.667
$ taskset -c 1 ./ticks
Ticks per usec: 345.043
$ taskset -c 1 ./ticks
Ticks per usec: 166.054
$ taskset -c 1 ./ticks
Ticks per usec: 256
$ taskset -c 1 ./ticks
Ticks per usec: 345.043
$ taskset -c 1 ./ticks
Ticks per usec: 256
$ taskset -c 1 ./ticks
Ticks per usec: 330.667
$ taskset -c 1 ./ticks
Ticks per usec: 256
$ taskset -c 1 ./ticks
Ticks per usec: 330.667
$ taskset -c 1 ./ticks
Ticks per usec: 330.667
$ taskset -c 1 ./ticks
Ticks per usec: 345.043
$ taskset -c 1 ./ticks
Ticks per usec: 256
$ taskset -c 1 ./ticks
Ticks per usec: 125.388
$ taskset -c 1 ./ticks
Ticks per usec: 360.727
$ taskset -c 1 ./ticks
Ticks per usec: 345.043

As we can see the difference between program executions can be up to 3 times (125-360). Such instability is not appropriate for any measurements.

Here is the code (gcc 4.9.3, running on Oracle Linux 6.6, kernel 3.8.13-55.1.2.el6uek.x86_64):

// g++ -O3 -std=c++11 -Wall ticks.cpp -o ticks
#include <x86intrin.h>
#include <ctime>
#include <cstdint>
#include <iostream>

int main()
{       
    timespec start, end;
    uint64_t s = 0;

    const double rdtsc_ticks_per_usec = [&]()
    {
        unsigned int dummy;

        clock_gettime(CLOCK_MONOTONIC, &start);

        uint64_t rd_start = __rdtscp(&dummy);
        for (size_t i = 0; i < 1000000; ++i) ++s;
        uint64_t rd_end = __rdtscp(&dummy);

        clock_gettime(CLOCK_MONOTONIC, &end);

        double usec_dur = double(end.tv_sec) * 1E6 + end.tv_nsec / 1E3;
        usec_dur -= double(start.tv_sec) * 1E6 + start.tv_nsec / 1E3;

        return (double)(rd_end - rd_start) / usec_dur;
    }();

    std::cout << s << std::endl;
    std::cout << "Ticks per usec: " << rdtsc_ticks_per_usec << std::endl;
    return 0;
}

When I run very similar program under Windows 7, i7-4470, VS2015 the result of calibration is pretty stable, small difference in last digit only.

So the question – what is that issue about? Is it CPU issue, Linux issue or my code issue?

Answer

Definitely it was my code (or gcc) issue. Compiler optimized out the loop replacing it with s = 1000000.

To prevent gcc to optimize this calibrating loop shall be changed this way:

for (size_t i = 0; i < 1000000; ++i) s += i;

Or more simple and correct way (thanks to Hal):

for (volatile size_t i = 0; i < 1000000; ++i);

Advertisement

Answer