SQL Server Diagnostic Information Queries Detailed, Day 28

For Day 28 of this series, we start out with Query #63, which is Overall Index Usage – Reads. This query retrieves information from the sys.indexes object catalog view, and the sys.dm_db_index_usage_stats dynamic management view about the overall index usage in the current database, ordered by reads. Query #63 is shown in Figure 1.

   1: --- Index Read/Write stats (all tables in current DB) ordered by Reads  (Query 63) (Overall Index Usage - Reads)

   2: SELECT OBJECT_NAME(i.[object_id]) AS [ObjectName], i.name AS [IndexName], i.index_id, 

   3:        s.user_seeks, s.user_scans, s.user_lookups,

   4:        s.user_seeks + s.user_scans + s.user_lookups AS [Total Reads], 

   5:        s.user_updates AS [Writes],  

   6:        i.type_desc AS [Index Type], i.fill_factor AS [Fill Factor], i.has_filter, i.filter_definition, 

   7:        s.last_user_scan, s.last_user_lookup, s.last_user_seek

   8: FROM sys.indexes AS i WITH (NOLOCK)

   9: LEFT OUTER JOIN sys.dm_db_index_usage_stats AS s WITH (NOLOCK)

  10: ON i.[object_id] = s.[object_id]

  11: AND i.index_id = s.index_id

  12: AND s.database_id = DB_ID()

  13: WHERE OBJECTPROPERTY(i.[object_id],'IsUserTable') = 1

  14: ORDER BY s.user_seeks + s.user_scans + s.user_lookups DESC OPTION (RECOMPILE); -- Order by reads

  15:  

  16:  

  17: -- Show which indexes in the current database are most active for Reads

Figure 1: Query #63 Overall Index Usage – Reads

This query shows you which indexes in the current database have the most cumulative reads (including seeks, scans and lookups) since the instance was last restarted or the index was created. This helps you understand your workload, and shows you which indexes are the most valuable for your workload.  Another use for this query is to help identify possible data compression candidates. If you find an index on a large table with a high number of reads and a low number of writes, then it might be a good candidate for data compression if the data is highly compressible.

 

Query #64 is Overall Index Usage – Writes. This query retrieves information from the sys.indexes object catalog view, and the sys.dm_db_index_usage_stats dynamic management view about the overall index usage in the current database, ordered by writes. Query #64 is shown in Figure 2.

   1: --- Index Read/Write stats (all tables in current DB) ordered by Writes  (Query 64) (Overall Index Usage - Writes)

   2: SELECT OBJECT_NAME(i.[object_id]) AS [ObjectName], i.name AS [IndexName], i.index_id,

   3:        s.user_updates AS [Writes], s.user_seeks + s.user_scans + s.user_lookups AS [Total Reads], 

   4:        i.type_desc AS [Index Type], i.fill_factor AS [Fill Factor], i.has_filter, i.filter_definition,

   5:        s.last_system_update, s.last_user_update

   6: FROM sys.indexes AS i WITH (NOLOCK)

   7: LEFT OUTER JOIN sys.dm_db_index_usage_stats AS s WITH (NOLOCK)

   8: ON i.[object_id] = s.[object_id]

   9: AND i.index_id = s.index_id

  10: AND s.database_id = DB_ID()

  11: WHERE OBJECTPROPERTY(i.[object_id],'IsUserTable') = 1

  12: ORDER BY s.user_updates DESC OPTION (RECOMPILE);                         -- Order by writes

  13:  

  14: -- Show which indexes in the current database are most active for Writes

Figure 2: Query #64 Overall Index Usage – Writes

This query shows you which indexes in the current database have the most cumulative writes since the instance was last restarted or the index was created. This helps you understand your workload, and shows you which indexes are the most volatile in your workload.  Another use for this query is to help identify possible indexes that you might consider dropping. If you find a non-clustered, non-key index on a table with a high number of writes and a very low number of reads, then you might want to drop that index, after doing some further investigation. You want to make sure that your instance has been running long enough so that you have seen your complete workload so that you don’t drop an index that is actually needed for queries that have not run yet.

SQL Server Diagnostic Information Queries Detailed, Day 27

For Day 27 of this series, we start out with Query #61, which is Volatile Indexes. This query retrieves information from the sys.objects object catalog view, the sys.stats object catalog view, and the sys.dm_db_stats_properties dynamic management function about the most volatile indexes and statistics in the current database. Query #61 is shown in Figure 1.

   1: -- Look at most frequently modified indexes and statistics (Query 61) (Volatile Indexes)

   2: SELECT o.name AS [Object Name], o.[object_id], o.type_desc, s.name AS [Statistics Name], 

   3:        s.stats_id, s.no_recompute, s.auto_created, 

   4:        sp.modification_counter, sp.rows, sp.rows_sampled, sp.last_updated

   5: FROM sys.objects AS o WITH (NOLOCK)

   6: INNER JOIN sys.stats AS s WITH (NOLOCK)

   7: ON s.object_id = o.object_id

   8: CROSS APPLY sys.dm_db_stats_properties(s.object_id, s.stats_id) AS sp

   9: WHERE o.type_desc NOT IN (N'SYSTEM_TABLE', N'INTERNAL_TABLE')

  10: AND sp.modification_counter > 0

  11: ORDER BY sp.modification_counter DESC, o.name OPTION (RECOMPILE);

Figure 1: Query #61 Volatile Indexes

This query shows you which indexes and statistics have had the most updates, which helps you understand your workload in more detail. Understanding which tables, indexes and statistics are most volatile is useful when you are thinking about how to layout your database files, how to design and configure your storage subsystem, and how to handle your index tuning and maintenance.

 

Query #62 is Index Fragmentation. This query retrieves information from the sys.dm_db_index_physical_stats dynamic management function and the sys.indexes object catalog view about the fragmentation status for all of the indexes larger than 2500 pages in the current database. Query #62 is shown in Figure 2.

   1: -- Get fragmentation info for all indexes above a certain size in the current database  (Query 62) (Index Fragmentation)

   2: -- Note: This query could take some time on a very large database

   3: SELECT DB_NAME(ps.database_id) AS [Database Name], OBJECT_NAME(ps.OBJECT_ID) AS [Object Name], 

   4: i.name AS [Index Name], ps.index_id, ps.index_type_desc, ps.avg_fragmentation_in_percent, 

   5: ps.fragment_count, ps.page_count, i.fill_factor, i.has_filter, i.filter_definition

   6: FROM sys.dm_db_index_physical_stats(DB_ID(),NULL, NULL, NULL , N'LIMITED') AS ps

   7: INNER JOIN sys.indexes AS i WITH (NOLOCK)

   8: ON ps.[object_id] = i.[object_id] 

   9: AND ps.index_id = i.index_id

  10: WHERE ps.database_id = DB_ID()

  11: AND ps.page_count > 2500

  12: ORDER BY ps.avg_fragmentation_in_percent DESC OPTION (RECOMPILE);

  13:  

  14: -- Helps determine whether you have framentation in your relational indexes

  15: -- and how effective your index maintenance strategy is

Figure 2: Query #62 Index Fragmentation

This query shows you the current fragmentation status for your heap tables and all indexes above 2500 pages in the current database. This helps you evaluate your index maintenance effectiveness. It is also useful when you are thinking about how you do your index maintenance and whether you might want to consider changing the fill factor on some indexes to reduce how often you need to rebuild or reorganize them.

I would say that most of the customers that I see fall into one of three categories regarding their index maintenance practices:

  1. Organizations with “Accidental DBAs” who don’t even know what index maintenance is, much less the difference between rebuild and reorganize, or about things like fill factors
    1. These organizations often have extremely high index fragmentation across most of their indexes and databases
  2. Organizations that are using the existing SQL Maintenance Plan Wizard (which is evil, BTW), which makes it far too easy for them to do redundant things like reorganize, then rebuild, then update statistics, etc.
    1. The organizations sometimes have less fragmentation, but they also often have a lot of extra transaction log activity, from redundant index maintenance activity
    2. The new SQL Maintenance Plan Wizard in SQL Server 2016 is a huge improvement, at least as far as the UI goes
  3. Organizations that are using something like Ola Hallengren’s free SQL Server Maintenance Solution for their index maintenance
    1. These organizations are usually in the best shape

SQL Server Diagnostic Information Queries Detailed, Day 26

For Day 26 of this series, we start out with Query #59, which is Table Properties. This query retrieves information from the sys.tables object catalog view, and the sys.partitions object catalog view about various table properties in the current database. Query #59 is shown in Figure 1.

   1: -- Get some key table properties (Query 59) (Table Properties)

   2: SELECT OBJECT_NAME(t.[object_id]) AS [ObjectName], p.[rows] AS [Table Rows], p.index_id, 

   3:        p.data_compression_desc AS [Index Data Compression],

   4:        t.create_date, t.lock_on_bulk_load, t.is_replicated, t.has_replication_filter, 

   5:        t.is_tracked_by_cdc, t.lock_escalation_desc, t.is_memory_optimized, t.durability_desc, t.is_filetable,

   6:        t.temporal_type_desc, t.is_remote_data_archive_enabled, t.remote_data_archive_migration_state_desc, t.is_external -- new for SQL Server 2016

   7: FROM sys.tables AS t WITH (NOLOCK)

   8: INNER JOIN sys.partitions AS p WITH (NOLOCK)

   9: ON t.[object_id] = p.[object_id]

  10: WHERE OBJECT_NAME(t.[object_id]) NOT LIKE N'sys%'

  11: ORDER BY OBJECT_NAME(t.[object_id]), p.index_id OPTION (RECOMPILE);

  12:  

  13: -- Gives you some good information about your tables

  14: -- Is Memory optimized and durability description are Hekaton-related properties that were new in SQL Server 2014

  15: -- temporal_type_desc, is_remote_data_archive_enabled, remote_data_archive_migration_state_desc, is_external are new in SQL Server 2016

Figure 1: Query #59 Table Properties

This query helps you understand what is going on with properties for the tables in the current database, showing you things such as whether they are being replicated, being tracked by change data capture, whether they are Hekaton tables, whether they are StretchDB tables, etc. It also shows you the data compression status for every index in each table. This can help you find possible data compression candidates.

 

Query #60 is Statistics Update. This query retrieves information from the sys.objects object catalog view, the sys.indexes object catalog view, the sys.stats object catalog view, and the sys.dm_db_partition_stats dynamic management view about the properties and status of the statistics in the current database. Query #60 is shown in Figure 2.

   1: -- When were Statistics last updated on all indexes?  (Query 60) (Statistics Update)

   2: SELECT SCHEMA_NAME(o.Schema_ID) + N'.' + o.NAME AS [Object Name], o.type_desc AS [Object Type],

   3:       i.name AS [Index Name], STATS_DATE(i.[object_id], i.index_id) AS [Statistics Date], 

   4:       s.auto_created, s.no_recompute, s.user_created, s.is_incremental, s.is_temporary,

   5:       st.row_count, st.used_page_count

   6: FROM sys.objects AS o WITH (NOLOCK)

   7: INNER JOIN sys.indexes AS i WITH (NOLOCK)

   8: ON o.[object_id] = i.[object_id]

   9: INNER JOIN sys.stats AS s WITH (NOLOCK)

  10: ON i.[object_id] = s.[object_id] 

  11: AND i.index_id = s.stats_id

  12: INNER JOIN sys.dm_db_partition_stats AS st WITH (NOLOCK)

  13: ON o.[object_id] = st.[object_id]

  14: AND i.[index_id] = st.[index_id]

  15: WHERE o.[type] IN ('U', 'V')

  16: AND st.row_count > 0

  17: ORDER BY STATS_DATE(i.[object_id], i.index_id) DESC OPTION (RECOMPILE);  

  18:  

  19: -- Helps discover possible problems with out-of-date statistics

  20: -- Also gives you an idea which indexes are the most active

Figure 2: Query #60 Statistics Update

This query shows you a number of relevant properties about the index-associated statistics in your current database, ordered by the last time that statistics were updated. This can help you determine whether you might have a problem with out of date statistics. My general guidance about statistics is that you should use the default database properties of auto create and auto update for statistics, plus I think you should also use the auto update statistics asynchronously database property, along with global trace flag 2371. In some situations, you may also want/need to do additional statistics maintenance on highly volatile tables with SQL Server Agent jobs.

SQL Server Diagnostic Information Queries Detailed, Day 25

For Day 25 of this series, we start out with Query #57, which is Buffer Usage. This query retrieves information from the sys.allocation_units object catalog view, the sys.dm_os_buffer_descriptors dynamic management view and the sys.partitions object catalog view about buffer pool usage in the current database. Query #57 is shown in Figure 1.

   1: -- Breaks down buffers used by current database by object (table, index) in the buffer cache  (Query 57) (Buffer Usage)

   2: -- Note: This query could take some time on a busy instance

   3: SELECT OBJECT_NAME(p.[object_id]) AS [Object Name], p.index_id, 

   4: CAST(COUNT(*)/128.0 AS DECIMAL(10, 2)) AS [Buffer size(MB)],  

   5: COUNT(*) AS [BufferCount], p.Rows AS [Row Count],

   6: p.data_compression_desc AS [Compression Type]

   7: FROM sys.allocation_units AS a WITH (NOLOCK)

   8: INNER JOIN sys.dm_os_buffer_descriptors AS b WITH (NOLOCK)

   9: ON a.allocation_unit_id = b.allocation_unit_id

  10: INNER JOIN sys.partitions AS p WITH (NOLOCK)

  11: ON a.container_id = p.hobt_id

  12: WHERE b.database_id = CONVERT(int,DB_ID())

  13: AND p.[object_id] > 100

  14: GROUP BY p.[object_id], p.index_id, p.data_compression_desc, p.[Rows]

  15: ORDER BY [BufferCount] DESC OPTION (RECOMPILE);

  16:  

  17: -- Tells you what tables and indexes are using the most memory in the buffer cache

  18: -- It can help identify possible candidates for data compression

Figure 1: Query #57 Buffer Usage

This query shows you which tables and indexes are using the most buffer pool space in the current database. This is very important information to understand if you are under internal memory pressure, or you are seeing high read latency for your data file(s). The query also displays the data compression status for the index.

If you see an index that is using a lot of space in the SQL Server buffer pool (because it shows up near the top of this query), then I would investigate whether SQL Server data compression might make sense for that index. What you want to look for are indexes that are highly compressible, on relatively static data, at least as far as UPDATES are concerned.

 

Query #58 is Table Sizes. This query retrieves information from the the sys.partitions object catalog view about the table sizes and clustered index (or heap) data compression status in the current database. Query #58 is shown in Figure 2.

   1: -- Get Table names, row counts, and compression status for clustered index or heap  (Query 58) (Table Sizes)

   2: SELECT OBJECT_NAME(object_id) AS [ObjectName], 

   3: SUM(Rows) AS [RowCount], data_compression_desc AS [CompressionType]

   4: FROM sys.partitions WITH (NOLOCK)

   5: WHERE index_id < 2 --ignore the partitions from the non-clustered index if any

   6: AND OBJECT_NAME(object_id) NOT LIKE N'sys%'

   7: AND OBJECT_NAME(object_id) NOT LIKE N'queue_%' 

   8: AND OBJECT_NAME(object_id) NOT LIKE N'filestream_tombstone%' 

   9: AND OBJECT_NAME(object_id) NOT LIKE N'fulltext%'

  10: AND OBJECT_NAME(object_id) NOT LIKE N'ifts_comp_fragment%'

  11: AND OBJECT_NAME(object_id) NOT LIKE N'filetable_updates%'

  12: AND OBJECT_NAME(object_id) NOT LIKE N'xml_index_nodes%'

  13: AND OBJECT_NAME(object_id) NOT LIKE N'sqlagent_job%'  

  14: AND OBJECT_NAME(object_id) NOT LIKE N'plan_persist%'  

  15: GROUP BY object_id, data_compression_desc

  16: ORDER BY SUM(Rows) DESC OPTION (RECOMPILE);

  17:  

  18: -- Gives you an idea of table sizes, and possible data compression opportunities

Figure 2: Query #58 Table Sizes

This query simply shows you the row counts and data compression status for the clustered index or heap for each table in the current database. I use this query to look for tables that might be good candidates of SQL Server data compression. Again, what you are looking for are large tables, that are relatively static, that compress well.

SQL Server Diagnostic Information Queries Detailed, Day 24

For Day 24 of this series, we start out with Query #54, which is Bad NC Indexes. This query retrieves information from the sys.dm_db_index_usage_stats dynamic management view and the sys.indexes object catalog view about non-clustered indexes that have more writes than reads in the current database. Query #54 is shown in Figure 1.

   1: -- Possible Bad NC Indexes (writes > reads)  (Query 54) (Bad NC Indexes)

   2: SELECT OBJECT_NAME(s.[object_id]) AS [Table Name], i.name AS [Index Name], i.index_id, 

   3: i.is_disabled, i.is_hypothetical, i.has_filter, i.fill_factor,

   4: user_updates AS [Total Writes], user_seeks + user_scans + user_lookups AS [Total Reads],

   5: user_updates - (user_seeks + user_scans + user_lookups) AS [Difference]

   6: FROM sys.dm_db_index_usage_stats AS s WITH (NOLOCK)

   7: INNER JOIN sys.indexes AS i WITH (NOLOCK)

   8: ON s.[object_id] = i.[object_id]

   9: AND i.index_id = s.index_id

  10: WHERE OBJECTPROPERTY(s.[object_id],'IsUserTable') = 1

  11: AND s.database_id = DB_ID()

  12: AND user_updates > (user_seeks + user_scans + user_lookups)

  13: AND i.index_id > 1

  14: ORDER BY [Difference] DESC, [Total Writes] DESC, [Total Reads] ASC OPTION (RECOMPILE);

  15:  

  16: -- Look for indexes with high numbers of writes and zero or very low numbers of reads

  17: -- Consider your complete workload, and how long your instance has been running

  18: -- Investigate further before dropping an index!

Figure 1: Query #54 Bad NC Indexes

What you are looking for with this query are indexes that have high numbers of writes and very few or even zero reads. If you are paying the cost to maintain an index as the data changes in your table, but the index is never used for reads, then you are placing unneeded stress on your storage subsystem that is not providing any benefits to the system. Having unused indexes also makes your database larger, and makes index maintenance more time consuming and resource intensive.

One key point to keep in mind before you start dropping indexes that appear to be unused is how long your SQL Server instance has been running. Before you drop an index, consider whether you have seen your complete normal business cycle. Perhaps there are monthly reports that actually do use an index that normally does not see any read activity with your regular workload.

 

 

Query #55 is Missing Indexes. This query retrieves information from the sys.dm_db_missing_index_group_stats dynamic management view, the sys.dm_db_missing_index_groups dynamic management view, the sys.dm_db_missing_index_details dynamic management view and the sys.partitions catalog view about “missing” indexes that the SQL Server Query Optimizer thinks that it would like to have in the current database. Query #55 is shown in Figure 2.

   1: -- Missing Indexes for current database by Index Advantage  (Query 55) (Missing Indexes)

   2: SELECT DISTINCT CONVERT(decimal(18,2), user_seeks * avg_total_user_cost * (avg_user_impact * 0.01)) AS [index_advantage], 

   3: migs.last_user_seek, mid.[statement] AS [Database.Schema.Table],

   4: mid.equality_columns, mid.inequality_columns, mid.included_columns,

   5: migs.unique_compiles, migs.user_seeks, migs.avg_total_user_cost, migs.avg_user_impact,

   6: OBJECT_NAME(mid.[object_id]) AS [Table Name], p.rows AS [Table Rows]

   7: FROM sys.dm_db_missing_index_group_stats AS migs WITH (NOLOCK)

   8: INNER JOIN sys.dm_db_missing_index_groups AS mig WITH (NOLOCK)

   9: ON migs.group_handle = mig.index_group_handle

  10: INNER JOIN sys.dm_db_missing_index_details AS mid WITH (NOLOCK)

  11: ON mig.index_handle = mid.index_handle

  12: INNER JOIN sys.partitions AS p WITH (NOLOCK)

  13: ON p.[object_id] = mid.[object_id]

  14: WHERE mid.database_id = DB_ID() 

  15: ORDER BY index_advantage DESC OPTION (RECOMPILE);

  16:  

  17: -- Look at index advantage, last user seek time, number of user seeks to help determine source and importance

  18: -- SQL Server is overly eager to add included columns, so beware

  19: -- Do not just blindly add indexes that show up from this query!!!

Figure 2: Query #55 Missing Indexes

This query is very useful, but also very easy to misinterpret and misuse. I have seen many novice DBAs and developers use the results of this query to pretty badly over-index their databases, which affects their database size and hurts insert, update, and delete performance. I like to focus on the last_user_seek column, and see how long ago that was. Was it a few seconds or minutes ago, or was it days or weeks ago?

I then start looking at the user_seeks, avg_total_user_cost, and avg_user_impact columns to get a sense for how often SQL Server thinks it wants this proposed index, how expensive it is not to have the index, and how much the query optimizer thinks the cost of the query would be reduced if it did have this index that it is requesting.

Next, I’ll look at any other proposed indexes on the same table to see if I can come up with a wider, consolidated index that covers multiple requested indexes. Finally, I’ll look at the existing indexes on that table, and look at the index usage metrics for that table to have a better idea of whether a new index would be a good idea, based on the volatility of that table.

This query is very similar to Query #28, but this one is only for the current database. It also pulls back the number of rows in a table, which is useful information when you are considering creating a new index, especially when you are using SQL Server Standard Edition, which does not have online index operations.

 

Query #56 is Missing Index Warnings. This query retrieves information from the sys.dm_exec_cached_plans dynamic management view, the sys.dm_exec_query_plan dynamic management function about missing index warnings in the plan cache for the current database. Query #56 is shown in Figure 3.

   1: -- Find missing index warnings for cached plans in the current database  (Query 56) (Missing Index Warnings)

   2: -- Note: This query could take some time on a busy instance

   3: SELECT TOP(25) OBJECT_NAME(objectid) AS [ObjectName], 

   4:                query_plan, cp.objtype, cp.usecounts, cp.size_in_bytes

   5: FROM sys.dm_exec_cached_plans AS cp WITH (NOLOCK)

   6: CROSS APPLY sys.dm_exec_query_plan(cp.plan_handle) AS qp

   7: WHERE CAST(query_plan AS NVARCHAR(MAX)) LIKE N'%MissingIndex%'

   8: AND dbid = DB_ID()

   9: ORDER BY cp.usecounts DESC OPTION (RECOMPILE);

  10:  

  11: -- Helps you connect missing indexes to specific stored procedures or queries

  12: -- This can help you decide whether to add them or not

Figure 3: Query #56 Missing Index Warnings

This query (which can be time consuming on a busy instance with a large plan cache) shows you where you have cached query plans with missing index warnings. This is very helpful, since it can often help you tie requested “missing” indexes to a particular stored procedure or prepared query plan.

SQL Server Diagnostic Information Queries Detailed, Day 23

For Day 23 of this series, we start out with Query #52, which is SP Logical Writes. This query retrieves information from the sys.procedures object catalog view and the sys.dm_exec_procedure_stats dynamic management view about the cached stored procedures that have the highest cumulative total logical writes in the current database. Query #52 is shown in Figure 1.

   1: -- Top Cached SPs By Total Logical Writes (Query 52) (SP Logical Writes)

   2: -- Logical writes relate to both memory and disk I/O pressure 

   3: SELECT TOP(25) p.name AS [SP Name], qs.total_logical_writes AS [TotalLogicalWrites], 

   4: qs.total_logical_writes/qs.execution_count AS [AvgLogicalWrites], qs.execution_count,

   5: ISNULL(qs.execution_count/DATEDIFF(Minute, qs.cached_time, GETDATE()), 0) AS [Calls/Minute],

   6: qs.total_elapsed_time, qs.total_elapsed_time/qs.execution_count AS [avg_elapsed_time], 

   7: qs.cached_time

   8: FROM sys.procedures AS p WITH (NOLOCK)

   9: INNER JOIN sys.dm_exec_procedure_stats AS qs WITH (NOLOCK)

  10: ON p.[object_id] = qs.[object_id]

  11: WHERE qs.database_id = DB_ID()

  12: AND qs.total_logical_writes > 0

  13: ORDER BY qs.total_logical_writes DESC OPTION (RECOMPILE);

  14:  

  15: -- This helps you find the most expensive cached stored procedures from a write I/O perspective

  16: -- You should look at this if you see signs of I/O pressure or of memory pressure

Figure 1: Query #52 SP Logical Writes

This query lets you see which cached stored procedures have the highest number cumulative logical writes in this database. This helps you see which stored procedures are causing the most write I/O pressure for this database. If you are seeing any signs of high write I/O latency on your instance of SQL Server (and if this database is causing a lot of I/O activity, as shown in Query #31), then the results of this query can help you figure out which stored procedures are the biggest offenders.

 

Query #53 is Top IO Statements. This query retrieves information from the sys.dm_exec_query_stats dynamic management and the sys.dm_exec_sql_text dynamic management function about the cached query statements that have the highest average I/O activity in the current database. Query #53 is shown in Figure 2.

   1: -- Lists the top statements by average input/output usage for the current database  (Query 53) (Top IO Statements)

   2: SELECT TOP(50) OBJECT_NAME(qt.objectid, dbid) AS [SP Name],

   3: (qs.total_logical_reads + qs.total_logical_writes) /qs.execution_count AS [Avg IO], qs.execution_count AS [Execution Count],

   4: SUBSTRING(qt.,qs.statement_start_offset/2, 

   5:     (CASE 

   6:         WHEN qs.statement_end_offset = -1 

   7:      THEN LEN(CONVERT(nvarchar(max), qt.)) * 2 

   8:         ELSE qs.statement_end_offset 

   9:      END - qs.statement_start_offset)/2) AS [Query Text]    

  10: FROM sys.dm_exec_query_stats AS qs WITH (NOLOCK)

  11: CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) AS qt

  12: WHERE qt.[dbid] = DB_ID()

  13: ORDER BY [Avg IO] DESC OPTION (RECOMPILE);

  14:  

  15: -- Helps you find the most expensive statements for I/O by SP

Figure 2: Query #53 Top IO Statements

This query shows you which query statements (which are often inside of stored procedures are causing the highest average I/O activity in the current database. Again, if you are under internal memory pressure, or if you are seeing high I/O latency for reads or for writes, the results of this query can point you in the right direction for further investigation.

Perhaps you have a query that is doing a clustered index scan because it is missing a useful non-clustered index. Perhaps a query is pulling back more rows or columns of data than it really needs (although can be hard for you to confirm this as a DBA). Perhaps the table(s) that are involved in this query might have indexes that would be good candidates for SQL Server Data Compression. There are many, many possible issues and actions that you can investigate in this area!

SQL Server Diagnostic Information Queries Detailed, Day 22

For Day 22 of this series, we start out with Query #50, which is SP Logical Reads. This query retrieves information from the sys.procedures object catalog view and the sys.dm_exec_procedure_stats dynamic management view about the cached stored procedures that have the highest cumulative total logical reads in the current database. Query #50 is shown in Figure 1.

   1: -- Top Cached SPs By Total Logical Reads. Logical reads relate to memory pressure  (Query 50) (SP Logical Reads)

   2: SELECT TOP(25) p.name AS [SP Name], qs.total_logical_reads AS [TotalLogicalReads], 

   3: qs.total_logical_reads/qs.execution_count AS [AvgLogicalReads],qs.execution_count, 

   4: ISNULL(qs.execution_count/DATEDIFF(Minute, qs.cached_time, GETDATE()), 0) AS [Calls/Minute], 

   5: qs.total_elapsed_time, qs.total_elapsed_time/qs.execution_count 

   6: AS [avg_elapsed_time], qs.cached_time

   7: FROM sys.procedures AS p WITH (NOLOCK)

   8: INNER JOIN sys.dm_exec_procedure_stats AS qs WITH (NOLOCK)

   9: ON p.[object_id] = qs.[object_id]

  10: WHERE qs.database_id = DB_ID()

  11: ORDER BY qs.total_logical_reads DESC OPTION (RECOMPILE);

  12:  

  13: -- This helps you find the most expensive cached stored procedures from a memory perspective

  14: -- You should look at this if you see signs of memory pressure

Figure 1: Query #50 SP Logical Reads

This query lets you see which cached stored procedures have the highest number cumulative logical reads in this database. This helps you see which stored procedures are causing the most memory pressure (and indirectly, read I/O pressure) for this database. If you are seeing any signs of memory pressure on your instance of SQL Server (and if this database is using a lot of memory in the SQL Server Buffer pool, as shown in Query #32), then the results of this query can help you figure out which stored procedures are the biggest offenders.

 

Query #51 is SP Physical Reads. This query retrieves information from the sys.procedures object catalog view and the sys.dm_exec_procedure_stats dynamic management view about the cached stored procedures that have the highest cumulative total physical reads in the current database. Query #51 is shown in Figure 2.

   1: -- Top Cached SPs By Total Physical Reads. Physical reads relate to disk read I/O pressure  (Query 51) (SP Physical Reads)

   2: SELECT TOP(25) p.name AS [SP Name],qs.total_physical_reads AS [TotalPhysicalReads], 

   3: qs.total_physical_reads/qs.execution_count AS [AvgPhysicalReads], qs.execution_count, 

   4: qs.total_logical_reads,qs.total_elapsed_time, qs.total_elapsed_time/qs.execution_count 

   5: AS [avg_elapsed_time], qs.cached_time 

   6: FROM sys.procedures AS p WITH (NOLOCK)

   7: INNER JOIN sys.dm_exec_procedure_stats AS qs WITH (NOLOCK)

   8: ON p.[object_id] = qs.[object_id]

   9: WHERE qs.database_id = DB_ID()

  10: AND qs.total_physical_reads > 0

  11: ORDER BY qs.total_physical_reads DESC, qs.total_logical_reads DESC OPTION (RECOMPILE);

  12:  

  13: -- This helps you find the most expensive cached stored procedures from a read I/O perspective

  14: -- You should look at this if you see signs of I/O pressure or of memory pressure

Figure 2: Query #51 SP Physical Reads

This query lets you see which cached stored procedures have the highest number cumulative physical reads in this database. This helps you see which stored procedures are causing the most read I/O pressure (and indirectly, memory pressure) for this database. If you are seeing any signs of high read I/O latency on your instance of SQL Server (and if this database is causing a lot of I/O activity, as shown in Query #31), then the results of this query can help you figure out which stored procedures are the biggest offenders.

SQL Server Diagnostic Information Queries Detailed, Day 21

For Day 21 of this series, we start out with Query #48, which is SP Avg Elapsed Time. This query retrieves information from the sys.procedures object catalog view and the sys.dm_exec_procedure_stats dynamic management view about the cached stored procedures that have the highest average elapsed time in the current database. Query #48 is shown in Figure 1.

   1: -- Top Cached SPs By Avg Elapsed Time (Query 48) (SP Avg Elapsed Time)

   2: SELECT TOP(25) p.name AS [SP Name], qs.min_elapsed_time, qs.total_elapsed_time/qs.execution_count AS [avg_elapsed_time], 

   3: qs.max_elapsed_time, qs.last_elapsed_time, qs.total_elapsed_time, qs.execution_count, 

   4: ISNULL(qs.execution_count/DATEDIFF(Minute, qs.cached_time, GETDATE()), 0) AS [Calls/Minute], 

   5: qs.total_worker_time/qs.execution_count AS [AvgWorkerTime], 

   6: qs.total_worker_time AS [TotalWorkerTime], qs.cached_time

   7: FROM sys.procedures AS p WITH (NOLOCK)

   8: INNER JOIN sys.dm_exec_procedure_stats AS qs WITH (NOLOCK)

   9: ON p.[object_id] = qs.[object_id]

  10: WHERE qs.database_id = DB_ID()

  11: ORDER BY avg_elapsed_time DESC OPTION (RECOMPILE);

  12:  

  13: -- This helps you find high average elapsed time cached stored procedures that

  14: -- may be easy to optimize with standard query tuning techniques

Figure 1: Query #48 SP Avg Elapsed Time

This query gives you a chance to look like a super hero. It shows you the cached stored procedures that have the highest average elapsed time in the current database. This basically gives you a list of stored procedures to look at much more closely, to see if you can do any query optimization or index tuning to make them dramatically faster. If you are able to do your DBA magic and make a long-running stored procedure run much, much faster, people are going to notice, and perhaps think you are some sort of evil genius.

 

Query #49 is SP Worker Time. This query retrieves information from the sys.procedures object catalog view and the sys.dm_exec_procedure_stats dynamic management view about the cached stored procedures that have the highest cumulative worker time in the current database. Query #49 is shown in Figure 2.

   1: -- Top Cached SPs By Total Worker time. Worker time relates to CPU cost  (Query 49) (SP Worker Time)

   2: SELECT TOP(25) p.name AS [SP Name], qs.total_worker_time AS [TotalWorkerTime], 

   3: qs.total_worker_time/qs.execution_count AS [AvgWorkerTime], qs.execution_count, 

   4: ISNULL(qs.execution_count/DATEDIFF(Minute, qs.cached_time, GETDATE()), 0) AS [Calls/Minute],

   5: qs.total_elapsed_time, qs.total_elapsed_time/qs.execution_count 

   6: AS [avg_elapsed_time], qs.cached_time

   7: FROM sys.procedures AS p WITH (NOLOCK)

   8: INNER JOIN sys.dm_exec_procedure_stats AS qs WITH (NOLOCK)

   9: ON p.[object_id] = qs.[object_id]

  10: WHERE qs.database_id = DB_ID()

  11: ORDER BY qs.total_worker_time DESC OPTION (RECOMPILE);

  12:  

  13: -- This helps you find the most expensive cached stored procedures from a CPU perspective

  14: -- You should look at this if you see signs of CPU pressure

Figure 2: Query #49 SP Worker Time

This query shows you which cached stored procedures have the highest cumulative total worker time in the current database. Worker time means CPU cost. If your instance or server is under CPU pressure, than looking at the stored procedures that show up at the top of this diagnostic query should be a high priority. Even if you are not under sustained CPU pressure, keeping an eye on the top offenders on this query is a good idea. Quite often, you will find the same stored procedures showing up on several of these different “Top SP cost” queries, which means that the SP in question is expensive from multiple perspectives.

SQL Server Diagnostic Information Queries Detailed, Day 20

For Day 20 of this series, we start out with Query #46, which is Query Execution Counts. This query retrieves information from the sys.dm_exec_query_stats dynamic management view, the sys.dm_exec_sql_text dynamic management function, and the sys.dm_exec_query_plan dynamic management function about the most frequently executed cached queries in the current database. Query #46 is shown in Figure 1.

   1: -- Get most frequently executed queries for this database (Query 46) (Query Execution Counts)

   2: SELECT TOP(50) LEFT(t., 50) AS [Short Query Text], qs.execution_count AS [Execution Count],

   3: qs.total_logical_reads AS [Total Logical Reads],

   4: qs.total_logical_reads/qs.execution_count AS [Avg Logical Reads],

   5: qs.total_worker_time AS [Total Worker Time],

   6: qs.total_worker_time/qs.execution_count AS [Avg Worker Time], 

   7: qs.total_elapsed_time AS [Total Elapsed Time],

   8: qs.total_elapsed_time/qs.execution_count AS [Avg Elapsed Time], 

   9: qs.creation_time AS [Creation Time]

  10: --,t. AS [Complete Query Text], qp.query_plan AS [Query Plan] -- uncomment out these columns if not copying results to Excel

  11: FROM sys.dm_exec_query_stats AS qs WITH (NOLOCK)

  12: CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS t 

  13: CROSS APPLY sys.dm_exec_query_plan(plan_handle) AS qp 

  14: WHERE t.dbid = DB_ID()

  15: ORDER BY qs.execution_count DESC OPTION (RECOMPILE);

Figure 1: Query #46 Query Execution Counts

This query shows you which cached queries (which might be part of a stored procedure or not) are being called the most often. This is useful as a part of understanding the nature of your workload. Keep in mind that just because a query is called a lot does not necessarily mean that it is a key part of your workload. It might be, but it could be that it is not actually that expensive for individual calls or cumulatively. You will need to look at the other metrics for that query to determine that.

You may notice that I have one line of this query commented out. This is because Excel does not deal very well with large quantities of text or XML. If you are working with this in real time, you should probably uncomment that line, so you see the extra information that it retrieves.

 

Query #47 is SP Execution Counts. This query retrieves information from the sys.procedures object catalog view and the sys.dm_exec_procedure_stats dynamic management view about the most frequently executed cached stored procedures in the current database. Query #47 is shown in Figure 2.

   1: -- Top Cached SPs By Execution Count (Query 47) (SP Execution Counts)

   2: SELECT TOP(100) p.name AS [SP Name], qs.execution_count,

   3: ISNULL(qs.execution_count/DATEDIFF(Minute, qs.cached_time, GETDATE()), 0) AS [Calls/Minute],

   4: qs.total_worker_time/qs.execution_count AS [AvgWorkerTime], qs.total_worker_time AS [TotalWorkerTime],  

   5: qs.total_elapsed_time, qs.total_elapsed_time/qs.execution_count AS [avg_elapsed_time],

   6: qs.cached_time

   7: FROM sys.procedures AS p WITH (NOLOCK)

   8: INNER JOIN sys.dm_exec_procedure_stats AS qs WITH (NOLOCK)

   9: ON p.[object_id] = qs.[object_id]

  10: WHERE qs.database_id = DB_ID()

  11: ORDER BY qs.execution_count DESC OPTION (RECOMPILE);

  12:  

  13: -- Tells you which cached stored procedures are called the most often

  14: -- This helps you characterize and baseline your workload

Figure 2: Query #47 SP Execution Counts

This query shows you which stored procedures with cached query plans are being called the most often. This helps you understand the nature and magnitude of your workload. Ideally, you should have a general idea of what your normal workload looks like, in terms of how many calls/minute or per second you are seeing for your top stored procedures.

If this rate suddenly changes, you would want to investigate further to understand what might have happened. Understanding which stored procedures are called the most often, can also help you identify possible candidates for middle-tier caching.

SQL Server Diagnostic Information Queries Detailed, Day 19

After eighteen days of queries, we have made it through all of the instance-level queries in this set. Now, we move on to the database-specific queries in the set. For these queries, you need to be connected to a particular database that you are concerned with, rather than the master system database.

For Day 19 of this series, we start out with Query #44, which is File Sizes and Space. This query retrieves information from the sys.database_files system catalog view and the sys.data_spaces system catalog view about The sizes and available space for all of your database files. Query #44 is shown in Figure 1.

   1: -- Individual File Sizes and space available for current database  (Query 44) (File Sizes and Space)

   2: SELECT f.name AS [File Name] , f.physical_name AS [Physical Name], 

   3: CAST((f.size/128.0) AS DECIMAL(15,2)) AS [Total Size in MB],

   4: CAST(f.size/128.0 - CAST(FILEPROPERTY(f.name, 'SpaceUsed') AS int)/128.0 AS DECIMAL(15,2)) 

   5: AS [Available Space In MB], [file_id], fg.name AS [Filegroup Name],

   6: f.is_percent_growth, f.growth

   7: FROM sys.database_files AS f WITH (NOLOCK) 

   8: LEFT OUTER JOIN sys.data_spaces AS fg WITH (NOLOCK) 

   9: ON f.data_space_id = fg.data_space_id OPTION (RECOMPILE);

  10:  

  11: -- Look at how large and how full the files are and where they are located

  12: -- Make sure the transaction log is not full!!

Figure 1: Query #44 File Sizes and Space

This query lets you see how large each of your database files are, plus how much space is available in each of your database files. For data files, you can also see what file group each file is in. You can also see exactly where each file is located in the file system. This is all extremely useful information.

 

Query #45 is IO Stats By File. This query retrieves information from the sys.dm_io_virtual_file_stats dynamic management function and the sys.database_files system catalog view about the cumulative I/O usage by database file. Query #45 is shown in Figure 2.

   1: -- I/O Statistics by file for the current database  (Query 45) (IO Stats By File)

   2: SELECT DB_NAME(DB_ID()) AS [Database Name], df.name AS [Logical Name], vfs.[file_id], df.type_desc,

   3: df.physical_name AS [Physical Name], CAST(vfs.size_on_disk_bytes/1048576.0 AS DECIMAL(10, 2)) AS [Size on Disk (MB)],

   4: vfs.num_of_reads, vfs.num_of_writes, vfs.io_stall_read_ms, vfs.io_stall_write_ms,

   5: CAST(100. * vfs.io_stall_read_ms/(vfs.io_stall_read_ms + vfs.io_stall_write_ms) AS DECIMAL(10,1)) AS [IO Stall Reads Pct],

   6: CAST(100. * vfs.io_stall_write_ms/(vfs.io_stall_write_ms + vfs.io_stall_read_ms) AS DECIMAL(10,1)) AS [IO Stall Writes Pct],

   7: (vfs.num_of_reads + vfs.num_of_writes) AS [Writes + Reads], 

   8: CAST(vfs.num_of_bytes_read/1048576.0 AS DECIMAL(10, 2)) AS [MB Read], 

   9: CAST(vfs.num_of_bytes_written/1048576.0 AS DECIMAL(10, 2)) AS [MB Written],

  10: CAST(100. * vfs.num_of_reads/(vfs.num_of_reads + vfs.num_of_writes) AS DECIMAL(10,1)) AS [# Reads Pct],

  11: CAST(100. * vfs.num_of_writes/(vfs.num_of_reads + vfs.num_of_writes) AS DECIMAL(10,1)) AS [# Write Pct],

  12: CAST(100. * vfs.num_of_bytes_read/(vfs.num_of_bytes_read + vfs.num_of_bytes_written) AS DECIMAL(10,1)) AS [Read Bytes Pct],

  13: CAST(100. * vfs.num_of_bytes_written/(vfs.num_of_bytes_read + vfs.num_of_bytes_written) AS DECIMAL(10,1)) AS [Written Bytes Pct]

  14: FROM sys.dm_io_virtual_file_stats(DB_ID(), NULL) AS vfs

  15: INNER JOIN sys.database_files AS df WITH (NOLOCK)

  16: ON vfs.[file_id]= df.[file_id] OPTION (RECOMPILE);

  17:  

  18: -- This helps you characterize your workload better from an I/O perspective for this database

  19: -- It helps you determine whether you has an OLTP or DW/DSS type of workload

Figure 2: Query #45 IO Stats By File

This query lets you see all of the cumulative file activity for each of the files in the current database, since SQL Server was last started. This includes your normal workload activity, plus any other activity that touches your data and log files. This would include things like database backups, index maintenance, DBCC CHECKDB activity, and HA-related activity from things like transactional replication, database mirroring, and AlwaysOn AG-related activity.

Looking at the results of this query helps you understand what kind of I/O workload activity you are seeing on each of your database files. This helps you do a better job when it comes to designing and configuring your storage subsystem.