2.6.20.3-rt8 - DMA suffers excessive delay periodically

From: John
Date: Tue Mar 27 2007 - 12:33:33 EST


Hello everyone,

I'm runnning 2.6.20.3 patched with -rt8 (and glibc 2.3.6).
http://people.redhat.com/mingo/realtime-preempt/older/patch-2.6.20-rt8

I've written a program to highlight a phenomenon I don't understand.

This system includes a PCI board that provides data at ~38 Mbit/s.
I request data in lumps of 1316 bytes.
In other words, a new request should complete every 277 µs.

However, if I time stamp each packet as soon as I receive it, I notice that almost exactly every 2 seconds, the packet is received 30-100 microseconds late.

(The system is otherwise idle while I run my test.)

Now, I know that this is not a hard real-time system, so it's not the delay per se that I find confusing. What I don't understand, is why the delay happens periodically.

My application runs as a SCHED_RR task with priority 80. As far as I understand, this is even higher priority than the softirq handlers.

Is there something with an even higher priority that could delay the delivery of my data?

The other possibility is that the delivery of the data itself is delayed. I don't understand PCI and DMA. Is it possible that the DMA request is delayed by something?

I've provided my test case. cf strange_delays.cxx
I've also provided the output of a long run of the test case.

The driver source code is available here:
http://www.dektec.com/Products/LinuxSDK/Downloads/LinuxSDK.zip

AFAIU, they use DMA to transfer data from the board's SDRAM to my system's RAM.

Regards. #include <ctime>
#include <cstdio>
#include <sched.h> // sched_setscheduler
#include <asm/msr.h>
#include "DTAPI.h"

#define BUFFER_SIZE (188*7)
#define CPU_MHZ 1266.794
#define US_DELAY 40

double us_period = 0;

static void set_SCHED_RR()
{
struct sched_param param = { 0 };
param.sched_priority = 80;
if (sched_setscheduler(0, SCHED_RR, &param) != 0) perror("SCHED_RR");
}

static void init_hardware(DtDevice& Dev, TsInpChannel& TsIn)
{
int rate = 0;
DTAPI_RESULT res;

res = Dev.AttachToType(140);
if (res != DTAPI_OK) printf("AttachToType: %lu\n", res);

res = TsIn.Attach(&Dev);
if (res != DTAPI_OK) printf("Attach: %lu\n", res);

res = TsIn.GetTsRateBps(rate);
if (res != DTAPI_OK) printf("GetTsRateBps: %lu\n", res);
printf("INCOMING RATE = %d bit/s\n", rate);
us_period = 8e6*BUFFER_SIZE / rate;
printf("EXPECTED PERIOD = %f µs\n", us_period);

res = TsIn.SetRxMode(DTAPI_RXMODE_ST188);
if (res != DTAPI_OK) printf("SetRxMode: %lu\n", res);

res = TsIn.SetRxControl(DTAPI_RXCTRL_RCV);
if (res != DTAPI_OK) printf("SetRxControl: %lu\n", res);
}

int main(void)
{
char buf[BUFFER_SIZE];
DtDevice Dev;
TsInpChannel TsIn;

set_SCHED_RR();
init_hardware(Dev, TsIn);

unsigned prev_tsc, curr_tsc;
struct timespec prev_spec, curr_spec, last_spec;
for (int i=0; i< 20; ++i) clock_gettime(CLOCK_MONOTONIC, &prev_spec);

TsIn.Read(buf, sizeof buf);
rdtscl(prev_tsc);
clock_gettime(CLOCK_MONOTONIC, &prev_spec);
last_spec = prev_spec;

while ( 1 )
{
TsIn.Read(buf, sizeof buf);
rdtscl(curr_tsc);
clock_gettime(CLOCK_MONOTONIC, &curr_spec);
double tsc_usdiff = (curr_tsc - prev_tsc) / CPU_MHZ;
double spec_usdiff = (curr_spec.tv_sec-prev_spec.tv_sec)*1e6 + (curr_spec.tv_nsec-prev_spec.tv_nsec)/1e3;
if (spec_usdiff > us_period+US_DELAY)
{
printf("NOW=%ld.%09ld\n", curr_spec.tv_sec, curr_spec.tv_nsec);
double temp = (curr_spec.tv_sec-last_spec.tv_sec) + (curr_spec.tv_nsec-last_spec.tv_nsec)/1e9;
last_spec = curr_spec;
printf("LAST: %f s\n", temp);
printf("CLOCK: %f µs\n", spec_usdiff);
}
if (tsc_usdiff > us_period+US_DELAY)
{
printf("TSC: %f µs\n", tsc_usdiff);
puts("+++++");
}
prev_tsc = curr_tsc;
prev_spec = curr_spec;
}

return 0;
}
root@SEND:~# ./testDT
INCOMING RATE = 38014635 bit/s
EXPECTED PERIOD = 276.945971 µs
NOW=1666.120122929
LAST: 0.555567 s
CLOCK: 367.129000 µs
TSC: 367.082572 µs
+++++
NOW=1668.090123211
LAST: 1.970000 s
CLOCK: 334.835000 µs
TSC: 334.804238 µs
+++++
NOW=1670.160123151
LAST: 2.070000 s
CLOCK: 335.609000 µs
TSC: 335.535217 µs
+++++
NOW=1674.120122451
LAST: 3.959999 s
CLOCK: 334.091000 µs
TSC: 334.003003 µs
+++++
NOW=1678.090121455
LAST: 3.969999 s
CLOCK: 361.673000 µs
TSC: 361.624700 µs
+++++
NOW=1680.160122488
LAST: 2.070001 s
CLOCK: 363.477000 µs
TSC: 363.429255 µs
+++++
NOW=1684.120123378
LAST: 3.960001 s
CLOCK: 363.449000 µs
TSC: 363.394522 µs
+++++
NOW=1685.191517226
LAST: 1.071394 s
CLOCK: 340.397000 µs
TSC: 340.393150 µs
+++++
NOW=1685.192610458
LAST: 0.001093 s
CLOCK: 325.702000 µs
TSC: 325.719888 µs
+++++
NOW=1685.193446243
LAST: 0.000836 s
CLOCK: 330.233000 µs
TSC: 330.311006 µs
+++++
NOW=1685.195118349
LAST: 0.001672 s
CLOCK: 342.856000 µs
TSC: 342.837904 µs
+++++
NOW=1685.195948913
LAST: 0.000831 s
CLOCK: 340.441000 µs
TSC: 340.427094 µs
+++++
NOW=1685.201210116
LAST: 0.005261 s
CLOCK: 340.938000 µs
TSC: 340.928359 µs
+++++
NOW=1685.202303271
LAST: 0.001093 s
CLOCK: 327.605000 µs
TSC: 327.615224 µs
+++++
NOW=1685.205362027
LAST: 0.003059 s
CLOCK: 340.357000 µs
TSC: 340.345786 µs
+++++
NOW=1685.206455332
LAST: 0.001093 s
CLOCK: 324.288000 µs
TSC: 324.295031 µs
+++++
NOW=1685.212833345
LAST: 0.006378 s
CLOCK: 333.487000 µs
TSC: 333.492265 µs
+++++
NOW=1685.213926684
LAST: 0.001093 s
CLOCK: 318.932000 µs
TSC: 318.938201 µs
+++++
NOW=1685.214505746
LAST: 0.000579 s
CLOCK: 333.171000 µs
TSC: 333.258604 µs
+++++
NOW=1685.216433536
LAST: 0.001928 s
CLOCK: 333.743000 µs
TSC: 333.723557 µs
+++++
NOW=1685.220848453
LAST: 0.004415 s
CLOCK: 318.938000 µs
TSC: 318.891627 µs
+++++
NOW=1685.222813290
LAST: 0.001965 s
CLOCK: 345.159000 µs
TSC: 345.141357 µs
+++++
NOW=1685.223905947
LAST: 0.001093 s
CLOCK: 330.277000 µs
TSC: 330.273115 µs
+++++
NOW=1685.225833823
LAST: 0.001928 s
CLOCK: 318.322000 µs
TSC: 318.310633 µs
+++++
NOW=1685.231376727
LAST: 0.005543 s
CLOCK: 323.892000 µs
TSC: 323.894808 µs
+++++
NOW=1685.234142203
LAST: 0.002765 s
CLOCK: 319.118000 µs
TSC: 319.126077 µs
+++++
NOW=1686.090122357
LAST: 0.855980 s
CLOCK: 331.238000 µs
TSC: 331.177761 µs
+++++
NOW=1688.160123287
LAST: 2.070001 s
CLOCK: 331.995000 µs
TSC: 331.955314 µs
+++++
NOW=1692.120123368
LAST: 3.960000 s
CLOCK: 330.912000 µs
TSC: 330.865160 µs
+++++
NOW=1696.090121892
LAST: 3.969999 s
CLOCK: 359.348000 µs
TSC: 359.272305 µs
+++++
NOW=1698.160121646
LAST: 2.070000 s
CLOCK: 358.589000 µs
TSC: 358.531853 µs
+++++
NOW=1702.120122383
LAST: 3.960001 s
CLOCK: 358.541000 µs
TSC: 358.492383 µs
+++++
NOW=1704.090122974
LAST: 1.970001 s
CLOCK: 327.748000 µs
TSC: 327.659430 µs
+++++
NOW=1706.160122441
LAST: 2.069999 s
CLOCK: 326.747000 µs
TSC: 326.706631 µs
+++++
NOW=1710.120123557
LAST: 3.960001 s
CLOCK: 325.793000 µs
TSC: 325.757779 µs
+++++
NOW=1714.090122247
LAST: 3.969999 s
CLOCK: 355.158000 µs
TSC: 355.067201 µs
+++++
NOW=1715.191685988
LAST: 1.101564 s
CLOCK: 317.896000 µs
TSC: 317.830681 µs
+++++
NOW=1716.160122015
LAST: 0.968436 s
CLOCK: 355.047000 µs
TSC: 355.003260 µs
+++++
NOW=1720.120121988
LAST: 3.960000 s
CLOCK: 353.483000 µs
TSC: 353.449732 µs
+++++
NOW=1720.187956278
LAST: 0.067834 s
CLOCK: 342.370000 µs
TSC: 342.283749 µs
+++++
NOW=1720.189415938
LAST: 0.001460 s
CLOCK: 415.925000 µs
TSC: 415.806358 µs
+++++
NOW=1720.192956331
LAST: 0.003540 s
CLOCK: 356.406000 µs
TSC: 356.272606 µs
+++++
NOW=1720.197653223
LAST: 0.004697 s
CLOCK: 346.931000 µs
TSC: 346.874867 µs
+++++
NOW=1720.198317781
LAST: 0.000665 s
CLOCK: 456.076000 µs
TSC: 456.134147 µs
+++++
NOW=1720.218743715
LAST: 0.020426 s
CLOCK: 389.993000 µs
TSC: 389.881070 µs
+++++
NOW=1720.232527110
LAST: 0.013783 s
CLOCK: 327.039000 µs
TSC: 327.012127 µs
+++++
NOW=1720.235585559
LAST: 0.003058 s
CLOCK: 339.401000 µs
TSC: 339.382725 µs
+++++
NOW=1722.090122182
LAST: 1.854537 s
CLOCK: 321.382000 µs
TSC: 321.296912 µs
+++++
NOW=1724.160130560
LAST: 2.070008 s
CLOCK: 329.502000 µs
TSC: 329.440304 µs
+++++
NOW=1728.120122106
LAST: 3.959992 s
CLOCK: 321.359000 µs
TSC: 321.311121 µs
+++++
NOW=1730.120127010
LAST: 2.000005 s
CLOCK: 387.895000 µs
TSC: 387.853905 µs
+++++
NOW=1732.090122917
LAST: 1.969996 s
CLOCK: 352.475000 µs
TSC: 352.386418 µs
+++++
NOW=1734.160122358
LAST: 2.069999 s
CLOCK: 349.973000 µs
TSC: 349.884038 µs
+++++
NOW=1738.120122212
LAST: 3.960000 s
CLOCK: 350.231000 µs
TSC: 350.178482 µs
+++++
NOW=1740.090122859
LAST: 1.970001 s
CLOCK: 319.788000 µs
TSC: 319.739437 µs
+++++
NOW=1742.090126733
LAST: 2.000004 s
CLOCK: 383.387000 µs
TSC: 383.297521 µs
+++++
NOW=1742.160121976
LAST: 0.069995 s
CLOCK: 318.684000 µs
TSC: 318.596394 µs
+++++
NOW=1744.160127139
LAST: 2.000005 s
CLOCK: 384.691000 µs
TSC: 384.649754 µs
+++++
NOW=1746.120122445
LAST: 1.959995 s
CLOCK: 318.163000 µs
TSC: 318.114863 µs
+++++
NOW=1748.120127187
LAST: 2.000005 s
CLOCK: 383.782000 µs
TSC: 383.724583 µs
+++++
NOW=1750.090122127
LAST: 1.969995 s
CLOCK: 346.185000 µs
TSC: 346.137572 µs
+++++
NOW=1752.160122295
LAST: 2.070000 s
CLOCK: 345.982000 µs
TSC: 345.918121 µs
+++++
NOW=1756.120121978
LAST: 3.960000 s
CLOCK: 346.185000 µs
TSC: 346.128100 µs
+++++
NOW=1760.090127929
LAST: 3.970006 s
CLOCK: 381.758000 µs
TSC: 381.693472 µs
+++++
NOW=1762.160127932
LAST: 2.070000 s
CLOCK: 380.056000 µs
TSC: 380.002589 µs
+++++
NOW=1766.120127556
LAST: 3.960000 s
CLOCK: 380.118000 µs
TSC: 380.027850 µs
+++++
NOW=1768.090121866
LAST: 1.969994 s
CLOCK: 341.860000 µs
TSC: 341.819586 µs
+++++
NOW=1770.160121781
LAST: 2.070000 s
CLOCK: 341.433000 µs
TSC: 341.375156 µs
+++++
NOW=1774.120123841
LAST: 3.960002 s
CLOCK: 343.805000 µs
TSC: 343.716500 µs
+++++
NOW=1778.090127807
LAST: 3.970004 s
CLOCK: 377.467000 µs
TSC: 377.403114 µs
+++++
NOW=1780.160127605
LAST: 2.070000 s
CLOCK: 375.649000 µs
TSC: 375.582770 µs
+++++
NOW=1784.120128708
LAST: 3.960001 s
CLOCK: 376.853000 µs
TSC: 376.796069 µs
+++++
NOW=1786.090125978
LAST: 1.969997 s
CLOCK: 343.319000 µs
TSC: 343.227076 µs
+++++
NOW=1788.160125049
LAST: 2.069999 s
CLOCK: 340.543000 µs
TSC: 340.484720 µs
+++++
NOW=1792.120125960
LAST: 3.960001 s
CLOCK: 340.299000 µs
TSC: 340.254217 µs
+++++
NOW=1796.090123335
LAST: 3.969997 s
CLOCK: 369.100000 µs
TSC: 369.035534 µs
+++++
NOW=1798.160128083
LAST: 2.070005 s
CLOCK: 371.975000 µs
TSC: 371.927875 µs
+++++
NOW=1802.120121956
LAST: 3.959994 s
CLOCK: 366.443000 µs
TSC: 366.402114 µs
+++++
NOW=1804.090122577
LAST: 1.970001 s
CLOCK: 334.571000 µs
TSC: 334.474271 µs
+++++
NOW=1806.160121338
LAST: 2.069999 s
CLOCK: 332.673000 µs
TSC: 332.627089 µs
+++++
NOW=1810.120122596
LAST: 3.960001 s
CLOCK: 334.326000 µs
TSC: 334.297447 µs
+++++
NOW=1814.090122300
LAST: 3.970000 s
CLOCK: 362.736000 µs
TSC: 362.699855 µs
+++++
NOW=1816.160121711
LAST: 2.069999 s
CLOCK: 361.687000 µs
TSC: 361.634962 µs
+++++
NOW=1820.120122921
LAST: 3.960001 s
CLOCK: 361.910000 µs
TSC: 361.862308 µs
+++++
NOW=1822.090122871
LAST: 1.970000 s
CLOCK: 330.282000 µs
TSC: 330.193386 µs
+++++
NOW=1824.160122395
LAST: 2.070000 s
CLOCK: 329.787000 µs
TSC: 329.730801 µs
+++++
NOW=1828.120121649
LAST: 3.959999 s
CLOCK: 327.519000 µs
TSC: 327.483395 µs
+++++
NOW=1832.090122094
LAST: 3.970000 s
CLOCK: 358.341000 µs
TSC: 358.289509 µs
+++++
NOW=1834.160121416
LAST: 2.069999 s
CLOCK: 358.616000 µs
TSC: 358.572112 µs
+++++
NOW=1838.120121708
LAST: 3.960000 s
CLOCK: 357.956000 µs
TSC: 357.920072 µs
+++++
NOW=1840.090121749
LAST: 1.970000 s
CLOCK: 326.833000 µs
TSC: 326.760310 µs
+++++
NOW=1842.160122763
LAST: 2.070001 s
CLOCK: 327.312000 µs
TSC: 327.261575 µs
+++++
NOW=1846.120123922
LAST: 3.960001 s
CLOCK: 326.125000 µs
TSC: 326.093272 µs
+++++
NOW=1850.090123129
LAST: 3.969999 s
CLOCK: 356.491000 µs
TSC: 356.453378 µs
+++++
NOW=1852.160121498
LAST: 2.069998 s
CLOCK: 354.432000 µs
TSC: 354.395427 µs
+++++
NOW=1856.120121477
LAST: 3.960000 s
CLOCK: 352.402000 µs
TSC: 352.313004 µs
+++++
NOW=1858.090123534
LAST: 1.970002 s
CLOCK: 322.080000 µs
TSC: 321.997894 µs
+++++
NOW=1860.160123827
LAST: 2.070000 s
CLOCK: 323.990000 µs
TSC: 323.917701 µs
+++++
NOW=1864.120121735
LAST: 3.959998 s
CLOCK: 319.928000 µs
TSC: 319.871266 µs
+++++
NOW=1868.090122222
LAST: 3.970000 s
CLOCK: 351.742000 µs
TSC: 351.698066 µs
+++++
NOW=1870.160121758
LAST: 2.070000 s
CLOCK: 349.349000 µs
TSC: 349.322779 µs
+++++
NOW=1873.300127947
LAST: 3.140006 s
CLOCK: 324.076000 µs
TSC: 323.946908 µs
+++++
NOW=1874.120122758
LAST: 0.819995 s
CLOCK: 350.780000 µs
TSC: 350.720796 µs
+++++
NOW=1876.090121470
LAST: 1.969999 s
CLOCK: 318.294000 µs
TSC: 318.208801 µs
+++++
NOW=1878.090126176
LAST: 2.000005 s
CLOCK: 382.688000 µs
TSC: 382.630483 µs
+++++
NOW=1878.160122361
LAST: 0.069996 s
CLOCK: 318.850000 µs
TSC: 318.770850 µs
+++++
NOW=1880.160127058
LAST: 2.000005 s
CLOCK: 384.580000 µs
TSC: 384.521872 µs
+++++
NOW=1882.120122004
LAST: 1.959995 s
CLOCK: 317.635000 µs
TSC: 317.585180 µs
+++++
NOW=1884.120126690
LAST: 2.000005 s
CLOCK: 381.658000 µs
TSC: 381.613743 µs
+++++
NOW=1886.090122584
LAST: 1.969996 s
CLOCK: 347.806000 µs
TSC: 347.731360 µs
+++++
NOW=1888.160122190
LAST: 2.070000 s
CLOCK: 346.916000 µs
TSC: 346.851185 µs
+++++
NOW=1892.120122420
LAST: 3.960000 s
CLOCK: 345.036000 µs
TSC: 344.978742 µs
+++++
NOW=1896.090127276
LAST: 3.970005 s
CLOCK: 381.048000 µs
TSC: 380.973544 µs
+++++
NOW=1898.160126913
LAST: 2.070000 s
CLOCK: 380.192000 µs
TSC: 380.127314 µs
+++++
NOW=1902.120126914
LAST: 3.960000 s
CLOCK: 379.271000 µs
TSC: 379.193460 µs
+++++
NOW=1904.090122092
LAST: 1.969995 s
CLOCK: 343.269000 µs
TSC: 343.222339 µs
+++++
NOW=1906.160123954
LAST: 2.070002 s
CLOCK: 344.804000 µs
TSC: 344.761658 µs
+++++
NOW=1910.120122069
LAST: 3.959998 s
CLOCK: 340.510000 µs
TSC: 340.444461 µs
+++++
NOW=1914.090127781
LAST: 3.970006 s
CLOCK: 376.229000 µs
TSC: 376.162186 µs
+++++
NOW=1916.160128205
LAST: 2.070000 s
CLOCK: 376.150000 µs
TSC: 376.085615 µs
+++++
NOW=1920.120127752
LAST: 3.960000 s
CLOCK: 374.878000 µs
TSC: 374.825741 µs
+++++
NOW=1922.090126083
LAST: 1.969998 s
CLOCK: 342.914000 µs
TSC: 342.866322 µs
+++++
NOW=1924.160126442
LAST: 2.070000 s
CLOCK: 343.080000 µs
TSC: 343.027359 µs
+++++
NOW=1928.120127491
LAST: 3.960001 s
CLOCK: 342.900000 µs
TSC: 342.841851 µs
+++++
NOW=1932.090122549
LAST: 3.969995 s
CLOCK: 368.473000 µs
TSC: 368.400071 µs
+++++
NOW=1934.160128106
LAST: 2.070006 s
CLOCK: 371.852000 µs
TSC: 371.796046 µs
+++++
NOW=1938.120121962
LAST: 3.959994 s
CLOCK: 364.932000 µs
TSC: 364.884898 µs
+++++
NOW=1940.090122110
LAST: 1.970000 s
CLOCK: 335.255000 µs
TSC: 335.176043 µs
+++++
NOW=1942.160122610
LAST: 2.070000 s
CLOCK: 335.332000 µs
TSC: 335.283400 µs
+++++
NOW=1946.120122338
LAST: 3.960000 s
CLOCK: 332.694000 µs
TSC: 332.642087 µs
+++++
NOW=1950.090121848
LAST: 3.970000 s
CLOCK: 363.514000 µs
TSC: 363.486092 µs
+++++
NOW=1952.160122097
LAST: 2.070000 s
CLOCK: 361.869000 µs
TSC: 361.824417 µs
+++++
NOW=1956.120121762
LAST: 3.960000 s
CLOCK: 361.824000 µs
TSC: 361.772317 µs
+++++
NOW=1958.090121866
LAST: 1.970000 s
CLOCK: 330.843000 µs
TSC: 330.802798 µs
+++++
NOW=1960.160121967
LAST: 2.070000 s
CLOCK: 329.076000 µs
TSC: 329.040870 µs
+++++
NOW=1964.120122585
LAST: 3.960001 s
CLOCK: 328.850000 µs
TSC: 328.807209 µs
+++++
NOW=1968.090122545
LAST: 3.970000 s
CLOCK: 360.024000 µs
TSC: 359.948816 µs
+++++
NOW=1970.160122226
LAST: 2.070000 s
CLOCK: 357.739000 µs
TSC: 357.691937 µs
+++++
NOW=1974.120122583
LAST: 3.960000 s
CLOCK: 358.642000 µs
TSC: 358.595004 µs
+++++
NOW=1976.090122186
LAST: 1.970000 s
CLOCK: 327.077000 µs
TSC: 326.990024 µs
+++++
NOW=1978.160121775
LAST: 2.070000 s
CLOCK: 324.884000 µs
TSC: 324.843660 µs
+++++
NOW=1982.120122656
LAST: 3.960001 s
CLOCK: 324.800000 µs
TSC: 324.742618 µs
+++++
NOW=1986.090121501
LAST: 3.969999 s
CLOCK: 354.848000 µs
TSC: 354.795649 µs
+++++
NOW=1988.160122082
LAST: 2.070001 s
CLOCK: 354.940000 µs
TSC: 354.881693 µs
+++++
NOW=1992.120121909
LAST: 3.960000 s
CLOCK: 353.775000 µs
TSC: 353.703128 µs
+++++
NOW=1994.090122897
LAST: 1.970001 s
CLOCK: 323.582000 µs
TSC: 323.510373 µs
+++++
NOW=1996.160122903
LAST: 2.070000 s
CLOCK: 321.960000 µs
TSC: 321.907114 µs
+++++
NOW=2000.120122840
LAST: 3.960000 s
CLOCK: 320.807000 µs
TSC: 320.750651 µs
+++++
NOW=2002.120127323
LAST: 2.000004 s
CLOCK: 387.854000 µs
TSC: 387.801016 µs
+++++
NOW=2004.090121920
LAST: 1.969995 s
CLOCK: 349.820000 µs
TSC: 349.763261 µs
+++++
NOW=2006.160122180
LAST: 2.070000 s
CLOCK: 350.914000 µs
TSC: 350.850257 µs
+++++
NOW=2010.120121540
LAST: 3.959999 s
CLOCK: 349.295000 µs
TSC: 349.216210 µs
+++++
NOW=2012.090124280
LAST: 1.970003 s
CLOCK: 319.678000 µs
TSC: 319.589452 µs
+++++
NOW=2014.090127024
LAST: 2.000003 s
CLOCK: 383.359000 µs
TSC: 383.291995 µs
+++++
NOW=2014.160122264
LAST: 0.069995 s
CLOCK: 318.495000 µs
TSC: 318.406939 µs
+++++
NOW=2016.160127610
LAST: 2.000005 s
CLOCK: 383.458000 µs
TSC: 383.393038 µs
+++++
NOW=2018.120122519
LAST: 1.959995 s
CLOCK: 317.861000 µs
TSC: 317.804631 µs
+++++
NOW=2020.120126831
LAST: 2.000004 s
CLOCK: 381.109000 µs
TSC: 381.043800 µs
+++++
NOW=2022.090122816
LAST: 1.969996 s
CLOCK: 346.321000 µs
TSC: 346.248088 µs
+++++
NOW=2024.160121683
LAST: 2.069999 s
CLOCK: 346.327000 µs
TSC: 346.278874 µs
+++++
NOW=2028.120122694
LAST: 3.960001 s
CLOCK: 346.039000 µs
TSC: 346.005744 µs
+++++
NOW=2032.090127434
LAST: 3.970005 s
CLOCK: 379.750000 µs
TSC: 379.686042 µs
+++++
NOW=2034.160127532
LAST: 2.070000 s
CLOCK: 380.559000 µs
TSC: 380.500697 µs
+++++
NOW=2038.120127129
LAST: 3.960000 s
CLOCK: 378.025000 µs
TSC: 377.935955 µs
+++++
NOW=2040.090122292
LAST: 1.969995 s
CLOCK: 343.166000 µs
TSC: 343.102351 µs
+++++
NOW=2042.160122800
LAST: 2.070001 s
CLOCK: 343.248000 µs
TSC: 343.184448 µs
+++++
NOW=2046.120122225
LAST: 3.959999 s
CLOCK: 340.421000 µs
TSC: 340.374204 µs
+++++
NOW=2050.090127308
LAST: 3.970005 s
CLOCK: 375.276000 µs
TSC: 375.218070 µs
+++++
NOW=2052.160127368
LAST: 2.070000 s
CLOCK: 376.382000 µs
TSC: 376.317696 µs
+++++
NOW=2056.120126565
LAST: 3.959999 s
CLOCK: 374.591000 µs
TSC: 374.537612 µs
+++++
NOW=2058.090125320
LAST: 1.969999 s
CLOCK: 339.915000 µs
TSC: 339.867413 µs
+++++
NOW=2060.160126928
LAST: 2.070002 s
CLOCK: 343.189000 µs
TSC: 343.141821 µs
+++++
NOW=2064.120121826
LAST: 3.959995 s
CLOCK: 336.745000 µs
TSC: 336.694048 µs
+++++
NOW=2068.090123131
LAST: 3.970001 s
CLOCK: 368.349000 µs
TSC: 368.282452 µs
+++++
NOW=2070.160123322
LAST: 2.070000 s
CLOCK: 368.078000 µs
TSC: 367.999059 µs
+++++
NOW=2074.120121390
LAST: 3.959998 s
CLOCK: 365.263000 µs
TSC: 365.173027 µs
+++++
NOW=2076.090122609
LAST: 1.970001 s
CLOCK: 333.969000 µs
TSC: 333.888541 µs
+++++
NOW=2078.160121934
LAST: 2.069999 s
CLOCK: 332.802000 µs
TSC: 332.713133 µs
+++++
NOW=2082.120122648
LAST: 3.960001 s
CLOCK: 333.954000 µs
TSC: 333.897224 µs
+++++
NOW=2086.090122233
LAST: 3.970000 s
CLOCK: 361.940000 µs
TSC: 361.900198 µs
+++++
NOW=2088.160122499
LAST: 2.070000 s
CLOCK: 361.707000 µs
TSC: 361.645224 µs
+++++
NOW=2092.120121260
LAST: 3.959999 s
CLOCK: 361.027000 µs
TSC: 360.979765 µs
+++++
NOW=2094.090122254
LAST: 1.970001 s
CLOCK: 330.781000 µs
TSC: 330.712807 µs
+++++
NOW=2096.160122449
LAST: 2.070000 s
CLOCK: 329.204000 µs
TSC: 329.155332 µs
+++++
NOW=2100.120131481
LAST: 3.960009 s
CLOCK: 337.284000 µs
TSC: 337.218995 µs
+++++
NOW=2104.090121454
LAST: 3.969990 s
CLOCK: 358.439000 µs
TSC: 358.358186 µs
+++++
NOW=2106.160121524
LAST: 2.070000 s
CLOCK: 358.088000 µs
TSC: 358.030587 µs
+++++
NOW=2110.120121778
LAST: 3.960000 s
CLOCK: 357.372000 µs
TSC: 357.324869 µs
+++++
NOW=2112.090122985
LAST: 1.970001 s
CLOCK: 326.026000 µs
TSC: 325.960653 µs
+++++
NOW=2114.160123826
LAST: 2.070001 s
CLOCK: 326.364000 µs
TSC: 326.282726 µs
+++++
NOW=2118.120122900
LAST: 3.959999 s
CLOCK: 324.595000 µs
TSC: 324.538165 µs
+++++
NOW=2122.090121704
LAST: 3.969999 s
CLOCK: 354.454000 µs
TSC: 354.397005 µs
+++++
NOW=2124.160122756
LAST: 2.070001 s
CLOCK: 355.006000 µs
TSC: 354.958265 µs
+++++
NOW=2128.120122622
LAST: 3.960000 s
CLOCK: 352.685000 µs
TSC: 352.627973 µs
+++++
NOW=2130.090123424
LAST: 1.970001 s
CLOCK: 323.587000 µs
TSC: 323.529319 µs
+++++
NOW=2132.160122126
LAST: 2.069999 s
CLOCK: 320.587000 µs
TSC: 320.511464 µs
+++++
NOW=2134.160126831
LAST: 2.000005 s
CLOCK: 387.588000 µs
TSC: 387.529464 µs
+++++
NOW=2136.120122321
LAST: 1.959995 s
CLOCK: 321.253000 µs
TSC: 321.195869 µs
+++++
NOW=2138.120127773
LAST: 2.000005 s
CLOCK: 387.593000 µs
TSC: 387.527096 µs
+++++
NOW=2140.090123895
LAST: 1.969996 s
CLOCK: 352.429000 µs
TSC: 352.365894 µs
+++++
NOW=2142.160122640
LAST: 2.069999 s
CLOCK: 350.855000 µs
TSC: 350.771317 µs
+++++
NOW=2146.120122168
LAST: 3.960000 s
CLOCK: 348.093000 µs
TSC: 348.060537 µs
+++++
NOW=2148.090123193
LAST: 1.970001 s
CLOCK: 319.125000 µs
TSC: 319.066083 µs
+++++
NOW=2150.090126482
LAST: 2.000003 s
CLOCK: 383.197000 µs
TSC: 383.160167 µs
+++++
NOW=2152.160128192
LAST: 2.070002 s
CLOCK: 384.931000 µs
TSC: 384.865258 µs
+++++
NOW=2156.120127494
LAST: 3.959999 s
CLOCK: 381.399000 µs
TSC: 381.342191 µs
+++++
NOW=2158.090121997
LAST: 1.969995 s
CLOCK: 346.070000 µs
TSC: 345.996271 µs
+++++
NOW=2160.160122440
LAST: 2.070000 s
CLOCK: 345.031000 µs
TSC: 344.984267 µs
+++++
NOW=2164.120122407
LAST: 3.960000 s
CLOCK: 345.267000 µs
TSC: 345.219507 µs
+++++
NOW=2168.090126618
LAST: 3.970004 s
CLOCK: 378.125000 µs
TSC: 378.077256 µs
+++++
NOW=2170.160126875
LAST: 2.070000 s
CLOCK: 377.927000 µs
TSC: 377.883065 µs
+++++
NOW=2174.120128092
LAST: 3.960001 s
CLOCK: 379.452000 µs
TSC: 379.405018 µs
+++++
NOW=2176.090123009
LAST: 1.969995 s
CLOCK: 343.414000 µs
TSC: 343.339170 µs
+++++
NOW=2178.160124020
LAST: 2.070001 s
CLOCK: 343.818000 µs
TSC: 343.738603 µs
+++++
NOW=2182.120124435
LAST: 3.960000 s
CLOCK: 343.208000 µs
TSC: 343.119718 µs
+++++
NOW=2186.090127466
LAST: 3.970003 s
CLOCK: 375.458000 µs
TSC: 375.395684 µs
+++++
NOW=2188.160127343
LAST: 2.070000 s
CLOCK: 375.499000 µs
TSC: 375.450942 µs
+++++
NOW=2192.120127347
LAST: 3.960000 s
CLOCK: 374.124000 µs
TSC: 374.066344 µs
+++++
NOW=2194.090126029
LAST: 1.969999 s
CLOCK: 340.386000 µs
TSC: 340.329998 µs
+++++
NOW=2196.160126484
LAST: 2.070000 s
CLOCK: 340.597000 µs
TSC: 340.549450 µs
+++++
NOW=2200.120122779
LAST: 3.959996 s
CLOCK: 337.022000 µs
TSC: 336.964810 µs
+++++
NOW=2204.090122370
LAST: 3.970000 s
CLOCK: 365.516000 µs
TSC: 365.466682 µs
+++++
NOW=2206.160122425
LAST: 2.070000 s
CLOCK: 366.506000 µs
TSC: 366.432111 µs
+++++
NOW=2210.120122424
LAST: 3.960000 s
CLOCK: 364.055000 µs
TSC: 364.006303 µs
+++++
NOW=2212.090122403
LAST: 1.970000 s
CLOCK: 334.248000 µs
TSC: 334.160092 µs
+++++
NOW=2214.160121998
LAST: 2.070000 s
CLOCK: 333.249000 µs
TSC: 333.193084 µs
+++++
NOW=2218.120122943
LAST: 3.960001 s
CLOCK: 332.062000 µs
TSC: 331.973470 µs
+++++
NOW=2222.090122671
LAST: 3.970000 s
CLOCK: 361.622000 µs
TSC: 361.563127 µs
+++++
NOW=2224.160121632
LAST: 2.069999 s
CLOCK: 361.439000 µs
TSC: 361.391039 µs
+++++
NOW=2228.120121316
LAST: 3.960000 s
CLOCK: 360.324000 µs
TSC: 360.286676 µs
+++++
NOW=2230.090123030
LAST: 1.970002 s
CLOCK: 328.910000 µs
TSC: 328.863256 µs
+++++
NOW=2232.160122704
LAST: 2.070000 s
CLOCK: 329.688000 µs
TSC: 329.641599 µs
+++++
NOW=2236.120123044
LAST: 3.960000 s
CLOCK: 328.803000 µs
TSC: 328.745637 µs
+++++
NOW=2240.090121969
LAST: 3.969999 s
CLOCK: 358.104000 µs
TSC: 358.048744 µs
+++++
NOW=2242.160122453
LAST: 2.070000 s
CLOCK: 358.032000 µs
TSC: 357.985592 µs
+++++
NOW=2246.120122246
LAST: 3.960000 s
CLOCK: 355.613000 µs
TSC: 355.577150 µs
+++++
NOW=2248.090122134
LAST: 1.970000 s
CLOCK: 324.386000 µs
TSC: 324.338448 µs
+++++
NOW=2250.160121764
LAST: 2.070000 s
CLOCK: 325.146000 µs
TSC: 325.072585 µs
+++++
NOW=2254.120122227
LAST: 3.960000 s
CLOCK: 324.315000 µs
TSC: 324.257930 µs
+++++
NOW=2258.090123537
LAST: 3.970001 s
CLOCK: 355.347000 µs
TSC: 355.306388 µs
+++++
NOW=2260.160121923
LAST: 2.069998 s
CLOCK: 352.091000 µs
TSC: 352.039874 µs
+++++
NOW=2264.120122712
LAST: 3.960001 s
CLOCK: 353.195000 µs
TSC: 353.147394 µs
+++++
NOW=2266.090122299
LAST: 1.970000 s
CLOCK: 321.688000 µs
TSC: 321.626879 µs
+++++
NOW=2268.090126962
LAST: 2.000005 s
CLOCK: 387.459000 µs
TSC: 387.416581 µs
+++++
NOW=2268.160122392
LAST: 0.069995 s
CLOCK: 319.917000 µs
TSC: 319.826270 µs
+++++
NOW=2270.160127275
LAST: 2.000005 s
CLOCK: 387.259000 µs
TSC: 387.201076 µs
+++++
NOW=2272.120121694
LAST: 1.959994 s
CLOCK: 318.380000 µs
TSC: 318.292477 µs
+++++
NOW=2274.120127225
LAST: 2.000006 s
CLOCK: 384.968000 µs
TSC: 384.903149 µs
+++++
NOW=2276.090122482
LAST: 1.969995 s
CLOCK: 350.154000 µs
TSC: 350.098753 µs
+++++
NOW=2278.160121859
LAST: 2.069999 s
CLOCK: 349.080000 µs
TSC: 349.036228 µs
+++++
NOW=2282.120122213
LAST: 3.960000 s
CLOCK: 348.486000 µs
TSC: 348.441025 µs
+++++
NOW=2284.090121948
LAST: 1.970000 s
CLOCK: 317.178000 µs
TSC: 317.104438 µs
+++++
NOW=2286.090127997
LAST: 2.000006 s
CLOCK: 384.011000 µs
TSC: 383.942456 µs
+++++
NOW=2286.160122193
LAST: 0.069994 s
CLOCK: 317.058000 µs
TSC: 316.986029 µs
+++++
NOW=2288.160126679
LAST: 2.000004 s
CLOCK: 382.487000 µs
TSC: 382.422083 µs
+++++
NOW=2292.120126736
LAST: 3.960000 s
CLOCK: 381.343000 µs
TSC: 381.278250 µs
+++++
NOW=2294.090122821
LAST: 1.969996 s
CLOCK: 346.514000 µs
TSC: 346.425701 µs
+++++
NOW=2296.160121637
LAST: 2.069999 s
CLOCK: 343.385000 µs
TSC: 343.343906 µs
+++++
NOW=2300.120122198
LAST: 3.960001 s
CLOCK: 343.108000 µs
TSC: 343.054988 µs
+++++
NOW=2304.090128313
LAST: 3.970006 s
CLOCK: 380.464000 µs
TSC: 380.382288 µs
+++++
NOW=2306.160127740
LAST: 2.069999 s
CLOCK: 379.400000 µs
TSC: 379.311869 µs
+++++
NOW=2310.120127246
LAST: 3.960000 s
CLOCK: 378.069000 µs
TSC: 378.033050 µs
+++++
NOW=2312.090122408
LAST: 1.969995 s
CLOCK: 340.708000 µs
TSC: 340.651282 µs
+++++
NOW=2314.160123453
LAST: 2.070001 s
CLOCK: 341.240000 µs
TSC: 341.180176 µs
+++++
NOW=2318.120123641
LAST: 3.960000 s
CLOCK: 340.593000 µs
TSC: 340.512348 µs
+++++
NOW=2322.090128388
LAST: 3.970005 s
CLOCK: 375.135000 µs
TSC: 375.077558 µs
+++++
NOW=2324.160127541
LAST: 2.069999 s
CLOCK: 373.726000 µs
TSC: 373.668489 µs
+++++
NOW=2328.120126737
LAST: 3.959999 s
CLOCK: 371.450000 µs
TSC: 371.401349 µs
+++++
NOW=2330.090127670
LAST: 1.970001 s
CLOCK: 343.017000 µs
TSC: 342.968944 µs
+++++
NOW=2332.160128197
LAST: 2.070001 s
CLOCK: 343.191000 µs
TSC: 343.148926 µs
+++++
NOW=2336.120122214
LAST: 3.959994 s
CLOCK: 336.162000 µs
TSC: 336.114633 µs
+++++
NOW=2340.090121624
LAST: 3.969999 s
CLOCK: 364.115000 µs
TSC: 364.050509 µs
+++++
NOW=2342.160122022
LAST: 2.070000 s
CLOCK: 365.283000 µs
TSC: 365.199867 µs
+++++
NOW=2346.120122459
LAST: 3.960000 s
CLOCK: 364.871000 µs
TSC: 364.833588 µs
+++++
NOW=2348.090122389
LAST: 1.970000 s
CLOCK: 332.302000 µs
TSC: 332.237128 µs
+++++
NOW=2350.160122731
LAST: 2.070000 s
CLOCK: 333.804000 µs
TSC: 333.733030 µs
+++++
NOW=2354.120122076
LAST: 3.959999 s
CLOCK: 330.538000 µs
TSC: 330.510722 µs
+++++
NOW=2358.090122410
LAST: 3.970000 s
CLOCK: 362.124000 µs
TSC: 362.077023 µs
+++++
NOW=2360.160122498
LAST: 2.070000 s
CLOCK: 360.274000 µs
TSC: 360.225104 µs
+++++
NOW=2364.120122635
LAST: 3.960000 s
CLOCK: 361.061000 µs
TSC: 361.008183 µs
+++++
NOW=2366.090122746
LAST: 1.970000 s
CLOCK: 329.509000 µs
TSC: 329.430042 µs
+++++
NOW=2368.160123306
LAST: 2.070001 s
CLOCK: 329.842000 µs
TSC: 329.789216 µs
+++++
NOW=2372.120122714
LAST: 3.959999 s
CLOCK: 326.468000 µs
TSC: 326.410608 µs
+++++
NOW=2376.090122940
LAST: 3.970000 s
CLOCK: 358.324000 µs
TSC: 358.236619 µs
+++++
NOW=2378.160122320
LAST: 2.069999 s
CLOCK: 357.320000 µs
TSC: 357.271980 µs
+++++
NOW=2382.120122416
LAST: 3.960000 s
CLOCK: 356.260000 µs
TSC: 356.212612 µs
+++++
NOW=2384.090123428
LAST: 1.970001 s
CLOCK: 326.208000 µs
TSC: 326.170632 µs
+++++
NOW=2386.160121986
LAST: 2.069999 s
CLOCK: 324.305000 µs
TSC: 324.232669 µs
+++++
NOW=2390.120123106
LAST: 3.960001 s
CLOCK: 323.174000 µs
TSC: 323.122781 µs
+++++
NOW=2394.090122864
LAST: 3.970000 s
CLOCK: 354.079000 µs
TSC: 354.033884 µs
+++++
NOW=2396.160122031
LAST: 2.069999 s
CLOCK: 352.640000 µs
TSC: 352.595607 µs
+++++
NOW=2400.120122434
LAST: 3.960000 s
CLOCK: 352.342000 µs
TSC: 352.294059 µs
+++++
NOW=2402.090122325
LAST: 1.970000 s
CLOCK: 320.273000 µs
TSC: 320.182287 µs
+++++
NOW=2404.090126776
LAST: 2.000004 s
CLOCK: 386.783000 µs
TSC: 386.716388 µs
+++++
NOW=2404.160121913
LAST: 0.069995 s
CLOCK: 319.872000 µs
TSC: 319.782853 µs
+++++
NOW=2406.160127550
LAST: 2.000006 s
CLOCK: 386.798000 µs
TSC: 386.740070 µs
+++++
NOW=2408.120122387
LAST: 1.959995 s
CLOCK: 317.538000 µs
TSC: 317.477822 µs
+++++
NOW=2410.120127165
LAST: 2.000005 s
CLOCK: 385.596000 µs
TSC: 385.529928 µs
+++++
NOW=2412.090122500
LAST: 1.969995 s
CLOCK: 348.127000 µs
TSC: 348.055801 µs
+++++
NOW=2414.160122346
LAST: 2.070000 s
CLOCK: 349.079000 µs
TSC: 349.036228 µs
+++++
NOW=2418.120122285
LAST: 3.960000 s
CLOCK: 347.996000 µs
TSC: 347.943707 µs
+++++
NOW=2420.090122689
LAST: 1.970000 s
CLOCK: 317.252000 µs
TSC: 317.195219 µs
+++++
NOW=2422.090127600
LAST: 2.000005 s
CLOCK: 383.134000 µs
TSC: 383.092279 µs
+++++
NOW=2424.160127034
LAST: 2.069999 s
CLOCK: 382.069000 µs
TSC: 382.010019 µs
+++++
NOW=2428.120127131
LAST: 3.960000 s
CLOCK: 381.331000 µs
TSC: 381.264831 µs
+++++
NOW=2430.090121976
LAST: 1.969995 s
CLOCK: 343.693000 µs
TSC: 343.620983 µs
+++++
NOW=2432.160122139
LAST: 2.070000 s
CLOCK: 343.227000 µs
TSC: 343.179712 µs
+++++
NOW=2436.120122465
LAST: 3.960000 s
CLOCK: 342.745000 µs
TSC: 342.704497 µs
+++++
NOW=2440.090127833
LAST: 3.970005 s
CLOCK: 377.911000 µs
TSC: 377.870435 µs
+++++
NOW=2442.160127236
LAST: 2.069999 s
CLOCK: 378.089000 µs
TSC: 378.052785 µs
+++++
NOW=2446.120127730
LAST: 3.960000 s
CLOCK: 376.282000 µs
TSC: 376.221390 µs
+++++
NOW=2448.090123683
LAST: 1.969996 s
CLOCK: 341.180000 µs
TSC: 341.099658 µs
+++++
NOW=2450.160125150
LAST: 2.070001 s
CLOCK: 343.364000 µs
TSC: 343.309962 µs
+++++
NOW=2454.120126427
LAST: 3.960001 s
CLOCK: 343.936000 µs
TSC: 343.868853 µs
+++++
NOW=2458.090128349
LAST: 3.970002 s
CLOCK: 375.221000 µs
TSC: 375.180179 µs
+++++
NOW=2460.160122322
LAST: 2.069994 s
CLOCK: 369.097000 µs
TSC: 369.065531 µs
+++++
NOW=2464.120127533
LAST: 3.960005 s
CLOCK: 371.685000 µs
TSC: 371.612906 µs
+++++
NOW=2466.090128807
LAST: 1.970001 s
CLOCK: 343.372000 µs
TSC: 343.309962 µs
+++++
NOW=2468.160123310
LAST: 2.069995 s
CLOCK: 337.366000 µs
TSC: 337.336615 µs
+++++
NOW=2472.120122060
LAST: 3.959999 s
CLOCK: 335.098000 µs
TSC: 335.055265 µs
+++++
NOW=2473.300099942
LAST: 1.179978 s
CLOCK: 342.201000 µs
TSC: 342.121134 µs
+++++
NOW=2476.090123406
LAST: 2.790023 s
CLOCK: 366.365000 µs
TSC: 366.312123 µs
+++++
NOW=2478.160122697
LAST: 2.069999 s
CLOCK: 365.226000 µs
TSC: 365.184868 µs
+++++
NOW=2482.120123255
LAST: 3.960001 s
CLOCK: 363.452000 µs
TSC: 363.400048 µs
+++++
NOW=2484.090123213
LAST: 1.970000 s
CLOCK: 333.372000 µs
TSC: 333.300442 µs
+++++
NOW=2486.160122411
LAST: 2.069999 s
CLOCK: 330.831000 µs
TSC: 330.803588 µs
+++++
NOW=2490.120123474
LAST: 3.960001 s
CLOCK: 332.284000 µs
TSC: 332.221340 µs
+++++
NOW=2494.090122536
LAST: 3.969999 s
CLOCK: 360.989000 µs
TSC: 360.901615 µs
+++++
NOW=2496.160121843
LAST: 2.069999 s
CLOCK: 359.949000 µs
TSC: 359.891190 µs
+++++
NOW=2500.120122412
LAST: 3.960001 s
CLOCK: 357.976000 µs
TSC: 357.927966 µs
+++++
NOW=2502.090122681
LAST: 1.970000 s
CLOCK: 327.334000 µs
TSC: 327.252892 µs
+++++
NOW=2504.160122755
LAST: 2.070000 s
CLOCK: 328.219000 µs
TSC: 328.162274 µs
+++++
NOW=2508.120123311
LAST: 3.960001 s
CLOCK: 327.850000 µs
TSC: 327.792838 µs
+++++
NOW=2512.090121835
LAST: 3.969999 s
CLOCK: 356.111000 µs
TSC: 356.042103 µs
+++++
NOW=2514.160123206
LAST: 2.070001 s
CLOCK: 355.766000 µs
TSC: 355.723188 µs
+++++
NOW=2518.120122660
LAST: 3.959999 s
CLOCK: 354.221000 µs
TSC: 354.172817 µs
+++++
NOW=2520.090121618
LAST: 1.969999 s
CLOCK: 323.293000 µs
TSC: 323.246716 µs
+++++
NOW=2522.160122367
LAST: 2.070001 s
CLOCK: 322.188000 µs
TSC: 322.115514 µs
+++++
NOW=2526.120121924
LAST: 3.960000 s
CLOCK: 322.128000 µs
TSC: 322.070518 µs
+++++
NOW=2530.090122578
LAST: 3.970001 s
CLOCK: 352.431000 µs
TSC: 352.402206 µs
+++++
NOW=2532.160122988
LAST: 2.070000 s
CLOCK: 352.322000 µs
TSC: 352.264851 µs
+++++
NOW=2536.120122437
LAST: 3.959999 s
CLOCK: 349.277000 µs
TSC: 349.239892 µs
+++++
NOW=2538.090122071
LAST: 1.970000 s
CLOCK: 319.431000 µs
TSC: 319.384209 µs
+++++
NOW=2540.090127355
LAST: 2.000005 s
CLOCK: 384.600000 µs
TSC: 384.527398 µs
+++++
NOW=2540.160123322
LAST: 0.069996 s
CLOCK: 318.959000 µs
TSC: 318.874261 µs
+++++
NOW=2542.160126917
LAST: 2.000004 s
CLOCK: 384.847000 µs
TSC: 384.789476 µs
+++++
NOW=2544.120122924
LAST: 1.959996 s
CLOCK: 318.917000 µs
TSC: 318.859262 µs
+++++
NOW=2546.120126784
LAST: 2.000004 s
CLOCK: 383.914000 µs
TSC: 383.825626 µs
+++++
NOW=2548.090122927
LAST: 1.969996 s
CLOCK: 348.772000 µs
TSC: 348.731522 µs
+++++
NOW=2550.160122457
LAST: 2.070000 s
CLOCK: 347.786000 µs
TSC: 347.745569 µs
+++++
NOW=2554.120123352
LAST: 3.960001 s
CLOCK: 346.456000 µs
TSC: 346.409124 µs
+++++
NOW=2558.090127640
LAST: 3.970004 s
CLOCK: 380.645000 µs
TSC: 380.574900 µs
+++++
NOW=2560.160127563
LAST: 2.070000 s
CLOCK: 379.975000 µs
TSC: 379.911019 µs
+++++
NOW=2564.120127688
LAST: 3.960000 s
CLOCK: 379.116000 µs
TSC: 379.058474 µs
+++++
NOW=2566.090122789
LAST: 1.969995 s
CLOCK: 342.936000 µs
TSC: 342.874216 µs
+++++
NOW=2568.160122498
LAST: 2.070000 s
CLOCK: 342.289000 µs
TSC: 342.248227 µs
+++++
NOW=2572.120123166
LAST: 3.960001 s
CLOCK: 342.993000 µs
TSC: 342.928685 µs
+++++
NOW=2576.090127848
LAST: 3.970005 s
CLOCK: 377.751000 µs
TSC: 377.692821 µs
+++++
NOW=2578.160126972
LAST: 2.069999 s
CLOCK: 376.376000 µs
TSC: 376.318486 µs
+++++
NOW=2582.120127229
LAST: 3.960000 s
CLOCK: 374.524000 µs
TSC: 374.459462 µs
+++++
NOW=2584.090124736
LAST: 1.969998 s
CLOCK: 340.645000 µs
TSC: 340.560502 µs
+++++
NOW=2586.160126703
LAST: 2.070002 s
CLOCK: 343.511000 µs
TSC: 343.466262 µs
+++++
NOW=2590.120127345
LAST: 3.960001 s
CLOCK: 343.315000 µs
TSC: 343.222339 µs
+++++
NOW=2594.090122521
LAST: 3.969995 s
CLOCK: 368.090000 µs
TSC: 368.025898 µs
+++++
NOW=2596.160128029
LAST: 2.070006 s
CLOCK: 371.835000 µs
TSC: 371.788152 µs
+++++
NOW=2600.120122367
LAST: 3.959994 s
CLOCK: 365.308000 µs
TSC: 365.271702 µs
+++++
NOW=2602.090122663
LAST: 1.970000 s
CLOCK: 334.398000 µs
TSC: 334.309288 µs
+++++
NOW=2604.160122401
LAST: 2.070000 s
CLOCK: 333.590000 µs
TSC: 333.532524 µs
+++++
NOW=2608.120122728
LAST: 3.960000 s
CLOCK: 332.624000 µs
TSC: 332.544992 µs
+++++
NOW=2612.090121961
LAST: 3.969999 s
CLOCK: 363.647000 µs
TSC: 363.599764 µs
+++++
NOW=2614.160122852
LAST: 2.070001 s
CLOCK: 362.402000 µs
TSC: 362.344627 µs
+++++
NOW=2618.120122643
LAST: 3.960000 s
CLOCK: 361.286000 µs
TSC: 361.245001 µs
+++++
NOW=2620.090123484
LAST: 1.970001 s
CLOCK: 332.233000 µs
TSC: 332.162135 µs
+++++
NOW=2622.160122755
LAST: 2.069999 s
CLOCK: 329.785000 µs
TSC: 329.713434 µs
+++++
NOW=2626.120122809
LAST: 3.960000 s
CLOCK: 330.158000 µs
TSC: 330.113657 µs
+++++
NOW=2630.090123340
LAST: 3.970001 s
CLOCK: 359.260000 µs
TSC: 359.191787 µs
+++++
NOW=2632.160121993
LAST: 2.069999 s
CLOCK: 358.628000 µs
TSC: 358.595794 µs
+++++
NOW=2636.120121742
LAST: 3.960000 s
CLOCK: 356.094000 µs
TSC: 356.057102 µs
+++++
NOW=2638.090122333
LAST: 1.970001 s
CLOCK: 326.517000 µs
TSC: 326.485601 µs
+++++
NOW=2640.160121823
LAST: 2.069999 s
CLOCK: 325.902000 µs
TSC: 325.820141 µs
+++++
NOW=2644.120122753
LAST: 3.960001 s
CLOCK: 325.498000 µs
TSC: 325.470440 µs
+++++
NOW=2648.090122795
LAST: 3.970000 s
CLOCK: 355.794000 µs
TSC: 355.732661 µs
+++++
NOW=2650.160122390
LAST: 2.070000 s
CLOCK: 353.454000 µs
TSC: 353.405526 µs
+++++
NOW=2654.120122565
LAST: 3.960000 s
CLOCK: 354.051000 µs
TSC: 354.002308 µs
+++++
NOW=2656.090121402
LAST: 1.969999 s
CLOCK: 321.640000 µs
TSC: 321.551886 µs
+++++
NOW=2658.090126534
LAST: 2.000005 s
CLOCK: 388.015000 µs
TSC: 387.957316 µs
+++++
NOW=2658.160122512
LAST: 0.069996 s
CLOCK: 320.978000 µs
TSC: 320.889584 µs
+++++
NOW=2662.120121702
LAST: 3.959999 s
CLOCK: 320.575000 µs
TSC: 320.518569 µs
+++++
NOW=2664.120127046
LAST: 2.000005 s
CLOCK: 386.974000 µs
TSC: 386.937418 µs
+++++
NOW=2666.090122442
LAST: 1.969995 s
CLOCK: 349.666000 µs
TSC: 349.598277 µs
+++++
NOW=2668.160122877
LAST: 2.070000 s
CLOCK: 351.366000 µs
TSC: 351.320736 µs
+++++
NOW=2672.120122157
LAST: 3.959999 s
CLOCK: 347.992000 µs
TSC: 347.933445 µs
+++++
NOW=2674.090121648
LAST: 1.969999 s
CLOCK: 317.323000 µs
TSC: 317.225216 µs
+++++
NOW=2676.090127443
LAST: 2.000006 s
CLOCK: 384.576000 µs
TSC: 384.507663 µs
+++++
NOW=2678.160126603
LAST: 2.069999 s
CLOCK: 383.345000 µs
TSC: 383.287259 µs
+++++
NOW=2680.120123631
LAST: 1.959997 s
CLOCK: 317.833000 µs
TSC: 317.781739 µs
+++++
NOW=2682.120127230
LAST: 2.000004 s
CLOCK: 381.622000 µs
TSC: 381.556907 µs
+++++
NOW=2684.090122917
LAST: 1.969996 s
CLOCK: 346.068000 µs
TSC: 346.004165 µs
+++++
NOW=2686.160122155
LAST: 2.069999 s
CLOCK: 346.160000 µs
TSC: 346.111522 µs
+++++
NOW=2690.120121752
LAST: 3.960000 s
CLOCK: 343.520000 µs
TSC: 343.431529 µs
+++++
NOW=2694.090127173
LAST: 3.970005 s
CLOCK: 378.671000 µs
TSC: 378.613255 µs
+++++
NOW=2696.160128135
LAST: 2.070001 s
CLOCK: 379.268000 µs
TSC: 379.202933 µs
+++++
NOW=2700.120127475
LAST: 3.959999 s
CLOCK: 377.713000 µs
TSC: 377.654930 µs
+++++
NOW=2702.090123317
LAST: 1.969996 s
CLOCK: 343.577000 µs
TSC: 343.487576 µs
+++++
NOW=2704.160122552
LAST: 2.069999 s
CLOCK: 341.080000 µs
TSC: 341.021508 µs
+++++
NOW=2708.120124301
LAST: 3.960002 s
CLOCK: 343.197000 µs
TSC: 343.143400 µs
+++++
NOW=2712.090127497
LAST: 3.970003 s
CLOCK: 376.181000 µs
TSC: 376.140083 µs
+++++
NOW=2714.160127365
LAST: 2.070000 s
CLOCK: 375.532000 µs
TSC: 375.443048 µs
+++++
NOW=2718.120128038
LAST: 3.960001 s
CLOCK: 375.309000 µs
TSC: 375.243331 µs
+++++
NOW=2720.090127647
LAST: 1.970000 s
CLOCK: 343.871000 µs
TSC: 343.813596 µs
+++++
NOW=2722.160126296
LAST: 2.069999 s
CLOCK: 340.581000 µs
TSC: 340.533662 µs
+++++
NOW=2726.120127543
LAST: 3.960001 s
CLOCK: 340.923000 µs
TSC: 340.885732 µs
+++++
NOW=2730.090122249
LAST: 3.969995 s
CLOCK: 365.460000 µs
TSC: 365.405899 µs
+++++
NOW=2732.160121938
LAST: 2.070000 s
CLOCK: 364.668000 µs
TSC: 364.614136 µs
+++++
NOW=2736.120123013
LAST: 3.960001 s
CLOCK: 364.880000 µs
TSC: 364.821747 µs
+++++
NOW=2738.090122141
LAST: 1.969999 s
CLOCK: 333.972000 µs
TSC: 333.915380 µs
+++++
NOW=2740.160122825
LAST: 2.070001 s
CLOCK: 334.412000 µs
TSC: 334.365335 µs
+++++
NOW=2744.120122636
LAST: 3.960000 s
CLOCK: 333.250000 µs
TSC: 333.202557 µs
+++++
NOW=2748.090122975
LAST: 3.970000 s
CLOCK: 363.450000 µs
TSC: 363.426887 µs
+++++
NOW=2750.160122921
LAST: 2.070000 s
CLOCK: 362.773000 µs
TSC: 362.714853 µs
+++++
NOW=2754.120122245
LAST: 3.959999 s
CLOCK: 361.122000 µs
TSC: 361.069756 µs
+++++
NOW=2756.090122961
LAST: 1.970001 s
CLOCK: 330.787000 µs
TSC: 330.706492 µs
+++++
NOW=2758.160123664
LAST: 2.070001 s
CLOCK: 330.875000 µs
TSC: 330.824112 µs
+++++
NOW=2762.120123202
LAST: 3.960000 s
CLOCK: 328.318000 µs
TSC: 328.277526 µs
+++++
NOW=2766.090121844
LAST: 3.969999 s
CLOCK: 356.731000 µs
TSC: 356.653884 µs
+++++
NOW=2768.160122899
LAST: 2.070001 s
CLOCK: 357.257000 µs
TSC: 357.215143 µs
+++++
NOW=2772.120122977
LAST: 3.960000 s
CLOCK: 356.305000 µs
TSC: 356.256818 µs
+++++
NOW=2774.090122256
LAST: 1.969999 s
CLOCK: 325.828000 µs
TSC: 325.787776 µs
+++++
NOW=2776.160122442
LAST: 2.070000 s
CLOCK: 324.267000 µs
TSC: 324.199515 µs
+++++
NOW=2780.120122083
LAST: 3.960000 s
CLOCK: 324.261000 µs
TSC: 324.212934 µs
+++++
NOW=2784.090122677
LAST: 3.970001 s
CLOCK: 354.735000 µs
TSC: 354.670136 µs
+++++
NOW=2786.160122276
LAST: 2.070000 s
CLOCK: 354.062000 µs
TSC: 354.025201 µs
+++++
NOW=2790.120122150
LAST: 3.960000 s
CLOCK: 351.429000 µs
TSC: 351.385466 µs
+++++
NOW=2792.090122058
LAST: 1.970000 s
CLOCK: 321.579000 µs
TSC: 321.539256 µs
+++++
NOW=2794.090127948
LAST: 2.000006 s
CLOCK: 388.562000 µs
TSC: 388.512260 µs
+++++
NOW=2794.160121497
LAST: 0.069994 s
CLOCK: 320.549000 µs
TSC: 320.469626 µs
+++++
NOW=2796.160127580
LAST: 2.000006 s
CLOCK: 387.558000 µs
TSC: 387.519991 µs
+++++
NOW=2798.120122338
LAST: 1.959995 s
CLOCK: 319.133000 µs
TSC: 319.096870 µs
+++++
NOW=2800.120127928
LAST: 2.000006 s
CLOCK: 385.816000 µs
TSC: 385.725698 µs
+++++
NOW=2802.090122314
LAST: 1.969994 s
CLOCK: 348.910000 µs
TSC: 348.860983 µs
+++++
NOW=2804.160122036
LAST: 2.070000 s
CLOCK: 348.124000 µs
TSC: 348.083430 µs
+++++
NOW=2808.120122575
LAST: 3.960001 s
CLOCK: 347.837000 µs
TSC: 347.771619 µs
+++++
NOW=2810.090123012
LAST: 1.970000 s
CLOCK: 317.003000 µs
TSC: 316.955243 µs
+++++
NOW=2812.090126646
LAST: 2.000004 s
CLOCK: 381.655000 µs
TSC: 381.590851 µs
+++++
NOW=2814.160127060
LAST: 2.070000 s
CLOCK: 381.551000 µs
TSC: 381.502438 µs
+++++
NOW=2818.120127361
LAST: 3.960000 s
CLOCK: 381.002000 µs
TSC: 380.945126 µs
+++++
NOW=2820.090121881
LAST: 1.969995 s
CLOCK: 344.411000 µs
TSC: 344.345647 µs
+++++
NOW=2822.160121366
LAST: 2.069999 s
CLOCK: 344.635000 µs
TSC: 344.583255 µs
+++++
NOW=2826.120122561
LAST: 3.960001 s
CLOCK: 344.959000 µs
TSC: 344.899802 µs
+++++
NOW=2830.090127456
LAST: 3.970005 s
CLOCK: 379.629000 µs
TSC: 379.581842 µs
+++++
NOW=2832.160127431
LAST: 2.070000 s
CLOCK: 377.934000 µs
TSC: 377.875961 µs
+++++
NOW=2836.120127194
LAST: 3.960000 s
CLOCK: 377.855000 µs
TSC: 377.811231 µs
+++++
NOW=2838.090122357
LAST: 1.969995 s
CLOCK: 340.677000 µs
TSC: 340.599182 µs
+++++
NOW=2840.160124089
LAST: 2.070002 s
CLOCK: 343.108000 µs
TSC: 343.049462 µs
+++++