Hello, This is on Windows 2008R2 server, i'm trying to benchmark my SAN, on NTFS and on raw device directly. I see huge difference in throughput between " -t32 -o1" Vs "-t1 -o32", later being significantly faster. Such difference in IOPs is not seen with raw device directly. Is there any foul play on NTFS part here? Thanks, Shekhar <<<<<<<<<<<<< NTFS numbers:: C:\Program Files (x86)\SQLIO>sqlio.exe -kR -t32 -frandom -s300 -b2 -BN -LS -o1 -Fparam.txt sqlio v1.5.SG using system counter for latency timings, 2604199 counts per second parameter file used: param.txt file F:\Testfile131072.dat with 1 thread (0) using mask 0x0 (0) 1 thread reading for 300 secs from file F:\Testfile131072.dat using 2KB random IOs enabling multiple I/Os per thread with 1 outstanding buffering set to not use file nor disk caches (as is SQL Server) using specified size: 131072 MB for file: F:\Testfile131072.dat initialization done interrupted by Ctrl-C - stopping I/Os now CUMULATIVE DATA: throughput metrics: IOs/sec: 238.13 MBs/sec: 0.46 latency metrics: Min_Latency(ms): 0 Avg_Latency(ms): 3 Max_Latency(ms): 228 histogram: ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+ %: 0 4 19 24 25 21 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 C:\Program Files (x86)\SQLIO>sqlio.exe -kR -t1 -frandom -s300 -b2 -BN -LS -o32 -Fparam.txt sqlio v1.5.SG using system counter for latency timings, 2604199 counts per second parameter file used: param.txt file F:\Testfile131072.dat with 1 thread (0) using mask 0x0 (0) 1 thread reading for 300 secs from file F:\Testfile131072.dat using 2KB random IOs enabling multiple I/Os per thread with 32 outstanding buffering set to not use file nor disk caches (as is SQL Server) using specified size: 131072 MB for file: F:\Testfile131072.dat initialization done interrupted by Ctrl-C - stopping I/Os now CUMULATIVE DATA: throughput metrics: IOs/sec: 2883.33 MBs/sec: 5.63 latency metrics: Min_Latency(ms): 1 Avg_Latency(ms): 10 Max_Latency(ms): 144 histogram: ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+ %: 0 1 7 10 11 10 7 6 5 5 4 3 3 3 2 2 2 2 1 1 1 1 1 1 10 <<<<<<<<<<<<<<<<<< Raw device numbers::: C:\Program Files (x86)\SQLIO>sqlio.exe -kR -t32 -frandom -s300 -b2 -BN -LS -o1 -R6 sqlio v1.5.SG using system counter for latency timings, 2604199 counts per second 32 threads reading for 300 secs from file 6: using 2KB random IOs enabling multiple I/Os per thread with 1 outstanding buffering set to not use file nor disk caches (as is SQL Server) using current size: 1126393 MB for file: 6: initialization done interrupted by Ctrl-C - stopping I/Os now CUMULATIVE DATA: throughput metrics: IOs/sec: 1865.08 MBs/sec: 3.64 latency metrics: Min_Latency(ms): 0 Avg_Latency(ms): 16 Max_Latency(ms): 441 histogram: ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+ %: 0 0 1 4 6 7 7 6 5 5 4 4 4 3 3 3 3 2 2 2 2 2 2 2 21 C:\Program Files (x86)\SQLIO>sqlio.exe -kR -t32 -frandom -s300 -b2 -BN -LS -o1 -R6 sqlio v1.5.SG using system counter for latency timings, 2604199 counts per second 32 threads reading for 300 secs from file 6: using 2KB random IOs enabling multiple I/Os per thread with 1 outstanding buffering set to not use file nor disk caches (as is SQL Server) using current size: 1126393 MB for file: 6: initialization done interrupted by Ctrl-C - stopping I/Os now CUMULATIVE DATA: throughput metrics: IOs/sec: 1865.08 MBs/sec: 3.64 latency metrics: Min_Latency(ms): 0 Avg_Latency(ms): 16 Max_Latency(ms): 441 histogram: ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+ %: 0 0 1 4 6 7 7 6 5 5 4 4 4 3 3 3 3 2 2 2 2 2 2 2 21 <<<<<<<<<<<<<