Comments on: SQL Server Index Maintenance Performance Tuning for Large Tables

By: Luke Campbell

Luke Campbell — Thu, 09 Feb 2012 21:36:55 +0000

Frank, I too ran into a similar issue. We currently defrag smaller tables throughout the day based on the number of pages and level of fragmentation. We had found it took longer to determine fragmentation than to actually reorganize or rebuild the indexes. I ran across a post by Paul Randal that described how to create a wrapper function for sys.dm_db_index_physical_stats that can be used with cross apply. So we now determine the page count of the indexes using sys.dm_db_partition_stats and then from there we cross apply to the wrapper function. Its been working like a charm for about 6 months now and has dramatically reduced our maintenance time for these small indexes. Check out the link below for more info. Hope it helps!

http://www.sqlskills.com/blogs/paul/post/Indexes-From-Every-Angle-Using-CROSS-APPLY-with-sysdm_db_index_physical_stats.aspx

By: Frank Kalis

Frank Kalis — Mon, 16 Jan 2012 07:59:21 +0000

One thing that comes into mind, might be to look into partitioning to split such beasts into manageable units. Apart from this… I honestly don’t know. Sometimes I think SQL Server is more sensitive to outdated statistics than it is to fragmentation. We had several cases where updating statistics fixed time-outs on the client with fragmentation staying the same.

I think this is more of a black art than a science.

By: Regan

Regan — Fri, 13 Jan 2012 10:56:17 +0000

Thanks for the article Frank – nice to see your approach on this.

We have got a similar, although slightly larger, issue to the 30 minute duration you talked about for the DMF – we have a 2 TB database, on a DL380 G6 (12 core) with 72 GB ram, where running the DMF in LIMITED mode takes 9h00+. SAMPLED was kiled after 26 hours, and DETAILEd – let’s not even go there (36 hours +). Large amount of data load activity takes place on the server (it is a performance data repository).

Given it is 2 TB in size, we don’t just want to reindex it all, but we do note that performance degrades if maintenance fails.

Is there any other metric peristed, even if it is somewhat out-of-date by thr time we do maintenance on indexes, or do we have to activel;y ask SQL to find out the fragmentation of an index by reviewing the strucutre (i.e. either SHOWCONTIG or the DMF?)

I’d like to avoid reindex 200 GB tables if they don’t need it, but we can’t afford 10 hours just to find out if we should do maintenance.

By: Ankit Singh

Ankit Singh — Sat, 24 Dec 2011 14:54:27 +0000

This is one of the best answer so far, I have read online.Just useful information. Very well presented. I had found another good collection of index in sql server over internet.
please check out this link…
http://mindstick.com/Articles/f7e21f40-d2f3-485a-b2aa-53093bac5ff7/?Index%20in%20SQL%20Server

Thanks

By: Frank Kalis

Frank Kalis — Tue, 29 Nov 2011 06:37:16 +0000

Do you have the link to that article?

By: Frank Kalis

Frank Kalis — Thu, 01 Sep 2011 06:27:11 +0000

Hi Colin,
I’m not sure I understand you fully. We had the idea of not bothering with detecting fragmentation on big tables at all, and go straight for a drop and (re)create, but never really investigated this much further when we implemented this. Since then, I thought about this as well and it is on my to-do list once we have reached our year-end change freeze. I guess that’ll go into a follow-up article to this one.

How big are your tables? And how many rows are inserted or deleted on average daily?

By: Frank Kalis

Frank Kalis — Thu, 01 Sep 2011 06:16:15 +0000

Hi Willem,

determining the number of processors, that is “cores” is quite straightforward:

EXEC sys.xp_msver ‘ProcessorCount’

We dump this into a table variable and use it later on. This gives you a count of the cores on the machine that runs SQL Server, not a “true” processor count. I guess I wouldn’t use this for licensing purposes.

By: Frank Kalis

Frank Kalis — Thu, 01 Sep 2011 06:11:39 +0000

Hi Matt,
I must confess, I haven’t thought about running sys.dm_db_index_physical_stats as a separate step outside the maintenance window and then “just” pick-up that results during the maintenance window. I guess that’ll give me food for thoughts now. Thanks!

By: Frank Kalis

Frank Kalis — Thu, 01 Sep 2011 06:05:51 +0000

Hi Bob,
thanks for the heads up on this issue with filtered indexes. I can’t wait to migrate that server to 2008 to try them and see whether they are useful for us or not.

By: LP

LP — Thu, 25 Aug 2011 14:10:30 +0000

I believe this is how it can be accomplished.
CREATE INDEX index_name
ON schema.table_name ({index cols})
WITH (MAXDOP=4);

See http://msdn.microsoft.com/en-gb/library/ms189329.aspx