{"id":1055,"date":"2008-04-28T23:05:37","date_gmt":"2008-04-28T23:05:37","guid":{"rendered":"\/blogs\/paul\/post\/SQL-Server-2008-Sparse-columns-and-XML-COLUMN_SET.aspx"},"modified":"2017-04-13T09:54:15","modified_gmt":"2017-04-13T16:54:15","slug":"sql-server-2008-sparse-columns-and-xml-column_set","status":"publish","type":"post","link":"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/","title":{"rendered":"SQL Server 2008: Sparse columns and XML COLUMN_SET"},"content":{"rendered":"<p><FONT face=Verdana size=2><br \/>\n<P>We&#8217;re sitting here in St. Pete Beach in Florida visiting some of Kimberly&#8217;s family and having some sun-kissed R&amp;R before heading back up to Seattle on Wednesday, and I thought I&#8217;d get the next post in my sparse columns mini-series out. Before I start though, Kimberly just posted the resources for the Accidental DBA class we taught at SQL Connections last week and in Iceland at the end of March &#8211; see <a href=\"http:\/\/www.sqlskills.com\/blogs\/kimberly\/catching-up-on-resources-the-accidental-dba-workshops\/\">here<\/a>.<\/P><br \/>\n<P>In my first post on sparse columns (see <a href=\"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/\">here<\/a>) I introduced the concepts and explained why sparse columns are a useful feature. In this post I&#8217;m going to use the example I gave &#8211; a document repository with 50 document types and 20 unique attributes per document-type. Yes, it&#8217;s a contrived example, but scale it up be a factor of 100+ (think Sharepoint Server) and methods like normalization no longer apply.<\/P><br \/>\n<P>I&#8217;m using a CTP-6 VPC on a Lenovo T60P laptop with 4GB, a dual-core 2.2GHz CPU, and the VPC is running off a 6200RPM drive. Your mileage may vary for run-times of the example scripts. The VPC is the one we gave out at SQL Connections and Iceland, and you can download the scripts and a VPC (maybe only CTP-5) from the Microsoft JumpStart site (see <a href=\"https:\/\/www.sqlskills.com\/blogs\/paul\/available-for-download-our-high-availability-hands-on-labs-and-sql-server-2008-jumpstart-materials\/\">here<\/a> for details).<\/P><br \/>\n<P>The first test I&#8217;ll do is just creating the schema necessary to store the 1000+ columns of information in the test scenario. I&#8217;ll do one with sparse columns and one without:<\/P><br \/>\n<BLOCKQUOTE dir=ltr style=\"MARGIN-RIGHT: 0px\"><FONT face=\"Courier New\" color=#008000 size=2><br \/>\n<P>&#8212; Create two tables, one with 1000 columns and one with 1000 columns but 997 sparse.<BR><\/FONT><FONT color=#0000ff><FONT face=\"Courier New\">CREATE<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> <\/FONT><FONT color=#0000ff>TABLE<\/FONT><FONT color=#000000> TableWithoutSparseColumns <\/FONT><FONT color=#808080>(<BR><\/FONT><\/FONT><FONT face=\"Courier New\">&nbsp;&nbsp;&nbsp;DocID <\/FONT><FONT face=\"Courier New\"><FONT color=#0000ff>INT<\/FONT> <FONT color=#0000ff>IDENTITY<\/FONT><FONT color=#808080>,<\/FONT> DocName <FONT color=#0000ff>VARCHAR<\/FONT> <FONT color=#808080>(<\/FONT>100<FONT color=#808080>),<\/FONT> DocType <FONT color=#0000ff>INT<\/FONT><FONT color=#808080>,<BR><\/FONT><\/FONT><FONT face=\"Courier New\">&nbsp;&nbsp;&nbsp;c0004 <\/FONT><FONT face=\"Courier New\"><FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0005 <FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0006 <FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0007 <FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0008 <FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0009 <FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL,<BR><\/FONT><\/FONT><FONT color=#808080><FONT face=\"Courier New\">&nbsp;&nbsp;&nbsp;&#8230;<BR><\/FONT><\/FONT><FONT face=\"Courier New\">&nbsp;&nbsp;&nbsp;c0994 <\/FONT><FONT face=\"Courier New\"><FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0995 <FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0996 <FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0997 <FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0998 <FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0999 <FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL,<BR><\/FONT><\/FONT><FONT face=\"Courier New\">&nbsp;&nbsp;&nbsp;c1000 <\/FONT><FONT face=\"Courier New\"><FONT color=#0000ff>INT<\/FONT> <FONT color=#808080>NULL);<BR><\/FONT><\/FONT><FONT face=\"Courier New\">GO<\/P><\/FONT><FONT color=#0000ff><br \/>\n<P><FONT face=\"Courier New\">CREATE<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> <\/FONT><FONT color=#0000ff>TABLE<\/FONT><FONT color=#000000> TableWithSparseColumns <\/FONT><FONT color=#808080>(<BR><\/FONT><\/FONT><FONT face=\"Courier New\">&nbsp;&nbsp;&nbsp;DocID <\/FONT><FONT face=\"Courier New\"><FONT color=#0000ff>INT<\/FONT> <FONT color=#0000ff>IDENTITY<\/FONT><FONT color=#808080>,<\/FONT> DocName <FONT color=#0000ff>VARCHAR<\/FONT> <FONT color=#808080>(<\/FONT>100<FONT color=#808080>),<\/FONT> DocType <FONT color=#0000ff>INT<\/FONT><FONT color=#808080>,<BR><\/FONT><\/FONT><FONT face=\"Courier New\">&nbsp;&nbsp;&nbsp;c0004 <\/FONT><FONT face=\"Courier New\"><FONT color=#0000ff>INT SPARSE<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0005 <FONT color=#0000ff>INT SPARSE<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0006 <FONT color=#0000ff>INT SPARSE<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0007 <FONT color=#0000ff>INT SPARSE<\/FONT> <FONT color=#808080>NULL,<BR><\/FONT><\/FONT><FONT color=#808080><FONT face=\"Courier New\">&nbsp;&nbsp;&nbsp;&#8230;<BR><\/FONT><\/FONT><FONT face=\"Courier New\">&nbsp;&nbsp;&nbsp;c0996 <\/FONT><FONT face=\"Courier New\"><FONT color=#0000ff>INT SPARSE<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0997 <FONT color=#0000ff>INT SPARSE<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0998 <FONT color=#0000ff>INT SPARSE<\/FONT> <FONT color=#808080>NULL,<\/FONT> c0999 <FONT color=#0000ff>INT SPARSE<\/FONT> <FONT color=#808080>NULL,<BR><\/FONT><\/FONT><FONT face=\"Courier New\">&nbsp;&nbsp;&nbsp;c1000 <\/FONT><FONT face=\"Courier New\"><FONT color=#0000ff>INT SPARSE<\/FONT> <FONT color=#808080>NULL);<BR><\/FONT><\/FONT><FONT face=\"Courier New\">GO<\/P><\/FONT><\/BLOCKQUOTE><br \/>\n<P>I won&#8217;t list all the column names for the sake of brevity. Next I&#8217;ll insert some values into each table (the same values in each table):<\/P><FONT face=\"Courier New\" color=#008000 size=2><br \/>\n<BLOCKQUOTE dir=ltr style=\"MARGIN-RIGHT: 0px\"><br \/>\n<P>&#8212; Insert a few rows in each<BR><\/FONT><FONT color=#0000ff><FONT face=\"Courier New\">INSERT<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> <\/FONT><FONT color=#0000ff>INTO<\/FONT><FONT color=#000000> TableWithSparseColumns <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#000000>DocName<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> Doctype<\/FONT><FONT color=#808080>)<\/FONT><FONT color=#000000> <\/FONT><FONT color=#0000ff>VALUES<\/FONT><FONT color=#000000> <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;aaaa&#8217;<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 1<\/FONT><FONT color=#808080>);<BR><\/FONT><\/FONT><FONT color=#0000ff><FONT face=\"Courier New\">INSERT<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> <\/FONT><FONT color=#0000ff>INTO<\/FONT><FONT color=#000000> TableWithSparseColumns <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#000000>DocName<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> Doctype<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> c0945<\/FONT><FONT color=#808080>)<\/FONT><FONT color=#000000> <\/FONT><FONT color=#0000ff>VALUES<\/FONT><FONT color=#000000> <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;bbbb&#8217;<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 2<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 46<\/FONT><FONT color=#808080>);<BR><\/FONT><\/FONT><FONT color=#0000ff><FONT face=\"Courier New\">INSERT<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> <\/FONT><FONT color=#0000ff>INTO<\/FONT><FONT color=#000000> TableWithSparseColumns <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#000000>DocName<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> Doctype<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> c0334<\/FONT><FONT color=#808080>)<\/FONT><FONT color=#000000> <\/FONT><FONT color=#0000ff>VALUES<\/FONT><FONT color=#000000> <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;cccc&#8217;<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 3<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 44<\/FONT><FONT color=#808080>);<BR><\/FONT><\/FONT><FONT color=#0000ff><FONT face=\"Courier New\">INSERT<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> <\/FONT><FONT color=#0000ff>INTO<\/FONT><FONT color=#000000> TableWithSparseColumns <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#000000>DocName<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> Doctype<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> c0233<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> c0234<\/FONT><FONT color=#808080>)<\/FONT><FONT color=#000000> <\/FONT><FONT color=#0000ff>VALUES<\/FONT><FONT color=#000000> <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;dddd&#8217;<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 4<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 12<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 34<\/FONT><FONT color=#808080>);<BR><\/FONT><\/FONT><FONT color=#0000ff><FONT face=\"Courier New\">INSERT<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> <\/FONT><FONT color=#0000ff>INTO<\/FONT><FONT color=#000000> TableWithSparseColumns <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#000000>DocName<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> Doctype<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> c0233<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> c0234<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000>c0235<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000>c0236<\/FONT><FONT color=#808080>)<\/FONT><FONT color=#000000> <\/FONT><FONT color=#0000ff>VALUES<\/FONT><FONT color=#000000> <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;eeee&#8217;<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 4<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 12<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 34<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 46<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 66<\/FONT><FONT color=#808080>);<BR><\/FONT><\/FONT><FONT face=\"Courier New\">GO<\/P><\/FONT><FONT color=#0000ff><br \/>\n<P><FONT face=\"Courier New\">INSERT<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> <\/FONT><FONT color=#0000ff>INTO<\/FONT><FONT color=#000000> TableWithoutSparseColumns <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#000000>DocName<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> Doctype<\/FONT><FONT color=#808080>)<\/FONT><FONT color=#000000> <\/FONT><FONT color=#0000ff>VALUES<\/FONT><FONT color=#000000> <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;aaaa&#8217;<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 1<\/FONT><FONT color=#808080>);<BR><\/FONT><\/FONT><FONT color=#0000ff><FONT face=\"Courier New\">INSERT<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> <\/FONT><FONT color=#0000ff>INTO<\/FONT><FONT color=#000000> TableWithoutSparseColumns <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#000000>DocName<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> Doctype<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> c0945<\/FONT><FONT color=#808080>)<\/FONT><FONT color=#000000> <\/FONT><FONT color=#0000ff>VALUES<\/FONT><FONT color=#000000> <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;bbbb&#8217;<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 2<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 46<\/FONT><FONT color=#808080>);<BR><\/FONT><\/FONT><FONT color=#0000ff><FONT face=\"Courier New\">INSERT<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> <\/FONT><FONT color=#0000ff>INTO<\/FONT><FONT color=#000000> TableWithoutSparseColumns <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#000000>DocName<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> Doctype<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> c0334<\/FONT><FONT color=#808080>)<\/FONT><FONT color=#000000> <\/FONT><FONT color=#0000ff>VALUES<\/FONT><FONT color=#000000> <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;cccc&#8217;<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 3<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 44<\/FONT><FONT color=#808080>);<BR><\/FONT><\/FONT><FONT color=#0000ff><FONT face=\"Courier New\">INSERT<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> <\/FONT><FONT color=#0000ff>INTO<\/FONT><FONT color=#000000> TableWithoutSparseColumns <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#000000>DocName<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> Doctype<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> c0233<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> c0234<\/FONT><FONT color=#808080>)<\/FONT><FONT color=#000000> <\/FONT><FONT color=#0000ff>VALUES<\/FONT><FONT color=#000000> <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;dddd&#8217;<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 4<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 12<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 34<\/FONT><FONT color=#808080>);<BR><\/FONT><\/FONT><FONT color=#0000ff><FONT face=\"Courier New\">INSERT<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> <\/FONT><FONT color=#0000ff>INTO<\/FONT><FONT color=#000000> TableWithoutSparseColumns <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#000000>DocName<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> Doctype<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> c0233<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> c0234<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000>c0235<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000>c0236<\/FONT><FONT color=#808080>)<\/FONT><FONT color=#000000> <\/FONT><FONT color=#0000ff>VALUES<\/FONT><FONT color=#000000> <\/FONT><FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;eeee&#8217;<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 4<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 12<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 34<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 46<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> 66<\/FONT><FONT color=#808080>);<BR><\/FONT><\/FONT><FONT face=\"Courier New\">GO<\/P><\/BLOCKQUOTE><\/FONT><br \/>\n<P>Now let&#8217;s see how big each table is:<\/P><FONT face=\"Courier New\" color=#008000 size=2><br \/>\n<BLOCKQUOTE dir=ltr style=\"MARGIN-RIGHT: 0px\"><br \/>\n<P>&#8212; Now lets see how big the rows are<BR><\/FONT><FONT color=#0000ff><FONT face=\"Courier New\">SELECT<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> [avg_record_size_in_bytes]<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> [page_count] <\/FONT><FONT color=#0000ff>FROM<\/FONT><FONT color=#000000> sys.dm_db_index_physical_stats <\/FONT><FONT color=#808080>(<BR><\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#ff00ff>&nbsp;&nbsp;&nbsp;DB_ID<\/FONT> <FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;SparseColumnsTest&#8217;<\/FONT><FONT color=#808080>),<\/FONT> <FONT color=#ff00ff>OBJECT_ID<\/FONT> <FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;TableWithoutSparseColumns&#8217;<\/FONT><FONT color=#808080>),<\/FONT> <FONT color=#808080>NULL,<\/FONT> <FONT color=#808080>NULL,<\/FONT> <FONT color=#ff0000>&#8216;DETAILED&#8217;<\/FONT><FONT color=#808080>);<\/P><\/FONT><\/FONT><FONT color=#0000ff><br \/>\n<P><FONT face=\"Courier New\">SELECT<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#000000> [avg_record_size_in_bytes]<\/FONT><FONT color=#808080>,<\/FONT><FONT color=#000000> [page_count] <\/FONT><FONT color=#0000ff>FROM<\/FONT><FONT color=#000000> sys.dm_db_index_physical_stats <\/FONT><FONT color=#808080>(<BR>&nbsp;&nbsp;&nbsp;<\/FONT><\/FONT><FONT face=\"Courier New\"><FONT color=#ff00ff>DB_ID<\/FONT> <FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;SparseColumnsTest&#8217;<\/FONT><FONT color=#808080>),<\/FONT> <FONT color=#ff00ff>OBJECT_ID<\/FONT> <FONT color=#808080>(<\/FONT><FONT color=#ff0000>&#8216;TableWithSparseColumns&#8217;<\/FONT><FONT color=#808080>),<\/FONT> <FONT color=#808080>NULL,<\/FONT> <FONT color=#808080>NULL,<\/FONT> <FONT color=#ff0000>&#8216;DETAILED&#8217;<\/FONT><FONT color=#808080>);<BR><\/FONT><\/FONT><FONT face=\"Courier New\" size=2>GO<\/FONT><\/P><br \/>\n<P><FONT face=\"Courier New\" size=2>avg_record_size_in_bytes page_count<BR>&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212; &#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8211;<BR>4135&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 5<\/FONT><\/P><br \/>\n<P><FONT face=\"Courier New\" size=2>(1 row(s) affected)<\/FONT><\/P><br \/>\n<P><FONT face=\"Courier New\" size=2>avg_record_size_in_bytes page_count<BR>&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212; &#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8211;<BR>40.6&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1<\/FONT><\/P><br \/>\n<P><FONT face=\"Courier New\" size=2>(1 row(s) affected)<\/FONT><FONT face=\"Courier New\" size=2><\/P><\/BLOCKQUOTE><\/FONT><br \/>\n<P>Ok &#8211; so that&#8217;s not a huge difference in page count (because we&#8217;ve only got 5 rows), but it&#8217;s a *massive* difference in average record size. Scaled up to hundreds of thousands or millions of records, the space savings will be astronomical!<\/P><br \/>\n<P>Now let&#8217;s try selecting the data back using results-to-grid mode and a simple <FONT face=\"Courier New\">SELECT *<\/FONT> statement. It takes 20 seconds to return &#8211; solely because the client still has to retrieve the metadata for 1000+ columns. Even though the columns are still defined as <FONT face=\"Courier New\">SPARSE<\/FONT>, they show up in a <FONT face=\"Courier New\">SELECT *<\/FONT> resultset, and that makes extracting out the non-NULL values pretty difficult&#8230;<\/P><br \/>\n<P>Time for another new feature &#8211; column sets. There&#8217;s a new column type available for use with sparse columns &#8211; an <FONT face=\"Courier New\">XML COLUMN_SET<\/FONT>. This is a column that is only materialized when selected, and will return all the non-NULL sparse columns in a row as an XML BLOB. It will also change the behavior of a SELECT * operation &#8211; removing all the sparse columns from the resultset and replacing them with itself, representing all the non-NULL sparse columns. Redefining our <FONT face=\"Courier New\">TableWithSparseColumns<\/FONT> to have an <FONT face=\"Courier New\">XML COLUMN_SET<\/FONT> column called <FONT face=\"Courier New\">SparseColumns <\/FONT><FONT face=Verdana>(using the syntax &#8216;<FONT face=\"Courier New\">SparseColumns <FONT color=#0000ff>XML<\/FONT> COLUMN_SET<FONT color=#0000ff> FOR<\/FONT> ALL_SPARSE_COLUMNS<\/FONT>&#8216;)<\/FONT>, and re-inserting the same values then gives the following results for a <FONT face=\"Courier New\">SELECT *<\/FONT> operation:<\/P><br \/>\n<BLOCKQUOTE dir=ltr style=\"MARGIN-RIGHT: 0px\"><br \/>\n<P><SPAN style=\"FONT-SIZE: 10pt; COLOR: #003300; LINE-HEIGHT: 120%; FONT-FAMILY: 'Verdana','sans-serif'; mso-fareast-font-family: 'Times New Roman'; mso-bidi-font-family: 'Courier New'; mso-fareast-theme-font: minor-fareast; mso-bidi-language: EN-US; mso-bidi-font-style: italic; mso-ansi-language: EN-US; mso-fareast-language: EN-US\"><FONT face=\"Courier New\">DocID&nbsp;&nbsp; DocName&nbsp;&nbsp; DocType&nbsp;&nbsp; SparseColumns<BR>&#8212;&#8212;- &#8212;&#8212;&#8212; &#8212;&#8212;&#8212; &#8212;&#8212;&#8212;&#8212;&#8212;<BR>1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; aaaa&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; NULL<BR>2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; bbbb&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;c0945&gt;46&lt;\/c0945&gt;<BR>3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; cccc&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;c0334&gt;44&lt;\/c0334&gt;<BR>4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; dddd&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;c0233&gt;12&lt;\/c0233&gt;&lt;c0234&gt;34&lt;\/c0234&gt;<BR>5&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; eeee&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;c0233&gt;12&lt;\/c0233&gt;&lt;c0234&gt;34&lt;\/c0234&gt;&lt;c0235&gt;46&lt;\/c0235&gt;&lt;c0236&gt;66&lt;\/c0236&gt;<\/FONT><\/SPAN><\/P><\/BLOCKQUOTE><br \/>\n<P>Pretty cool &#8211; and it returns virtually instantaneously (obviously scaling up to hundreds of thousands or millions of rows would take longer due to the time necessary to read the pages into the buffer pool). One downside is that the XML blob only returns the column name and value &#8211; not the datatype &#8211; but if your application can cope with that then not having to wade through hundreds (or thousands by RTM) of NULL columns values is great.<\/P><br \/>\n<P>Next time I&#8217;ll discuss the internals of how sparse columns are stored.<\/P><\/FONT><\/p>\n","protected":false},"excerpt":{"rendered":"<p>We&#8217;re sitting here in St. Pete Beach in Florida visiting some of Kimberly&#8217;s family and having some sun-kissed R&amp;R before heading back up to Seattle on Wednesday, and I thought I&#8217;d get the next post in my sparse columns mini-series out. Before I start though, Kimberly just posted the resources for the Accidental DBA class [&hellip;]<\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[38,83,86],"tags":[],"class_list":["post-1055","post","type-post","status-publish","format-standard","hentry","category-example-scripts","category-sparse-columns","category-sql-server-2008"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>SQL Server 2008: Sparse columns and XML COLUMN_SET - Paul S. Randal<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"SQL Server 2008: Sparse columns and XML COLUMN_SET - Paul S. Randal\" \/>\n<meta property=\"og:description\" content=\"We&#8217;re sitting here in St. Pete Beach in Florida visiting some of Kimberly&#8217;s family and having some sun-kissed R&amp;R before heading back up to Seattle on Wednesday, and I thought I&#8217;d get the next post in my sparse columns mini-series out. Before I start though, Kimberly just posted the resources for the Accidental DBA class [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/\" \/>\n<meta property=\"og:site_name\" content=\"Paul S. Randal\" \/>\n<meta property=\"article:published_time\" content=\"2008-04-28T23:05:37+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-04-13T16:54:15+00:00\" \/>\n<meta name=\"author\" content=\"Paul Randal\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Paul Randal\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/\",\"url\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/\",\"name\":\"SQL Server 2008: Sparse columns and XML COLUMN_SET - Paul S. Randal\",\"isPartOf\":{\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/#website\"},\"datePublished\":\"2008-04-28T23:05:37+00:00\",\"dateModified\":\"2017-04-13T16:54:15+00:00\",\"author\":{\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/#\/schema\/person\/ffcec826c18782e1e0adf173826a7fce\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"SQL Server 2008: Sparse columns and XML COLUMN_SET\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/#website\",\"url\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/\",\"name\":\"Paul S. Randal\",\"description\":\"In Recovery...\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/#\/schema\/person\/ffcec826c18782e1e0adf173826a7fce\",\"name\":\"Paul Randal\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/0b6a266bba2f088f2551ef529293001bd73bf026bc1908b9866728c062beeeb6?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/0b6a266bba2f088f2551ef529293001bd73bf026bc1908b9866728c062beeeb6?s=96&d=mm&r=g\",\"caption\":\"Paul Randal\"},\"sameAs\":[\"http:\/\/3.209.169.194\/blogs\/paul\"],\"url\":\"https:\/\/www.sqlskills.com\/blogs\/paul\/author\/paul\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"SQL Server 2008: Sparse columns and XML COLUMN_SET - Paul S. Randal","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/","og_locale":"en_US","og_type":"article","og_title":"SQL Server 2008: Sparse columns and XML COLUMN_SET - Paul S. Randal","og_description":"We&#8217;re sitting here in St. Pete Beach in Florida visiting some of Kimberly&#8217;s family and having some sun-kissed R&amp;R before heading back up to Seattle on Wednesday, and I thought I&#8217;d get the next post in my sparse columns mini-series out. Before I start though, Kimberly just posted the resources for the Accidental DBA class [&hellip;]","og_url":"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/","og_site_name":"Paul S. Randal","article_published_time":"2008-04-28T23:05:37+00:00","article_modified_time":"2017-04-13T16:54:15+00:00","author":"Paul Randal","twitter_misc":{"Written by":"Paul Randal","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/","url":"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/","name":"SQL Server 2008: Sparse columns and XML COLUMN_SET - Paul S. Randal","isPartOf":{"@id":"https:\/\/www.sqlskills.com\/blogs\/paul\/#website"},"datePublished":"2008-04-28T23:05:37+00:00","dateModified":"2017-04-13T16:54:15+00:00","author":{"@id":"https:\/\/www.sqlskills.com\/blogs\/paul\/#\/schema\/person\/ffcec826c18782e1e0adf173826a7fce"},"breadcrumb":{"@id":"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.sqlskills.com\/blogs\/paul\/sql-server-2008-sparse-columns-and-xml-column_set\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.sqlskills.com\/blogs\/paul\/"},{"@type":"ListItem","position":2,"name":"SQL Server 2008: Sparse columns and XML COLUMN_SET"}]},{"@type":"WebSite","@id":"https:\/\/www.sqlskills.com\/blogs\/paul\/#website","url":"https:\/\/www.sqlskills.com\/blogs\/paul\/","name":"Paul S. Randal","description":"In Recovery...","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.sqlskills.com\/blogs\/paul\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.sqlskills.com\/blogs\/paul\/#\/schema\/person\/ffcec826c18782e1e0adf173826a7fce","name":"Paul Randal","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.sqlskills.com\/blogs\/paul\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/0b6a266bba2f088f2551ef529293001bd73bf026bc1908b9866728c062beeeb6?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/0b6a266bba2f088f2551ef529293001bd73bf026bc1908b9866728c062beeeb6?s=96&d=mm&r=g","caption":"Paul Randal"},"sameAs":["http:\/\/3.209.169.194\/blogs\/paul"],"url":"https:\/\/www.sqlskills.com\/blogs\/paul\/author\/paul\/"}]}},"_links":{"self":[{"href":"https:\/\/www.sqlskills.com\/blogs\/paul\/wp-json\/wp\/v2\/posts\/1055","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.sqlskills.com\/blogs\/paul\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.sqlskills.com\/blogs\/paul\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.sqlskills.com\/blogs\/paul\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.sqlskills.com\/blogs\/paul\/wp-json\/wp\/v2\/comments?post=1055"}],"version-history":[{"count":0,"href":"https:\/\/www.sqlskills.com\/blogs\/paul\/wp-json\/wp\/v2\/posts\/1055\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.sqlskills.com\/blogs\/paul\/wp-json\/wp\/v2\/media?parent=1055"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.sqlskills.com\/blogs\/paul\/wp-json\/wp\/v2\/categories?post=1055"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.sqlskills.com\/blogs\/paul\/wp-json\/wp\/v2\/tags?post=1055"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}