{"id":737,"date":"2004-06-06T22:13:50","date_gmt":"2004-06-06T22:13:50","guid":{"rendered":"\/blogs\/kimberly\/post\/Stored-Procedures-are-NOT-evil-but-they-can-be-frustrating!.aspx"},"modified":"2013-01-02T07:07:50","modified_gmt":"2013-01-02T15:07:50","slug":"stored-procedures-are-not-evil-but-they-can-be-frustrating","status":"publish","type":"post","link":"https:\/\/www.sqlskills.com\/blogs\/kimberly\/stored-procedures-are-not-evil-but-they-can-be-frustrating\/","title":{"rendered":"Stored Procedures are NOT evil&#8230; but they can be frustrating!"},"content":{"rendered":"<p><DIV><FONT face=Verdana>After a group of <a href=\"http:\/\/www.microsoftregionaldirectors.com\/Public\/pubHome.aspx\" class=\"broken_link\">RDs<\/a> started chatting about stored procedures I finally felt like I could join in. Often they talk about .Net and client stuff and well, quite honestly, I don&#8217;t care what you do with the data&#8230;only that I serve it up quickly. Ok, I&#8217;m really kidding here but my primary focus is backend server tuning and availability, so when the group started talking about stored procedures, I had to chime in. It all started because someone had blogged that <EM>stored procedures are not precompiled and are therefore of no use<\/EM>&#8230; <STRONG>which is COMPLETELY wrong<\/STRONG>. However, the irony is that I don&#8217;t even believe that they should be compiled (and saved and therefore reused) ALL the time. In fact, a precompiled plan is also an optimized plan and that plan may or may not <STRONG>always<\/STRONG> be the best plan for every execution&#8230;&#8230;. So, here&#8217;s a bunch of stuff about sprocs. <STRONG>You definitely want to use them &#8211; but use them effectively!!!! <\/STRONG><\/FONT><\/DIV><br \/>\n<DIV><FONT face=Verdana><\/FONT>&nbsp;<\/DIV><br \/>\n<DIV><FONT face=Verdana size=3>Benefits of stored procedures:<\/FONT><\/DIV><br \/>\n<UL><br \/>\n<LI><FONT face=Verdana><FONT size=2>Centralized logic which can be changed with minimal client impact<\/FONT> <\/FONT><br \/>\n<LI><FONT face=Verdana><FONT size=2>Logic on the server so roundtrips are minimized<\/FONT> <\/FONT><br \/>\n<LI><FONT face=Verdana><FONT size=2>Compiled plans saved in cache. Not every plan is saved, not every plan should be saved. However, the default is that stored procedure plans&nbsp;are compiled and saved &#8211; on <EM>first<\/EM> exection. <\/FONT>The plan is NOT created when the stored procedure is <FONT size=2>created (I&#8217;ve also seen this written up incorrectly before) NOR is this plan saved permanently. There are many reasons for why the procedure&#8217;s plan may fall out of cache:<\/FONT><\/FONT><br \/>\n<UL><br \/>\n<LI><FONT face=Verdana><FONT size=2>Server restart<\/FONT><\/FONT><br \/>\n<LI><FONT face=Verdana><FONT size=2>Falls out of cache due to low re-use (and not enough cache to keep it around)<\/FONT><\/FONT><br \/>\n<LI><FONT face=Verdana><FONT size=2>Specifically being removed from cache by:<\/FONT><\/FONT><br \/>\n<UL><br \/>\n<LI><FONT face=Verdana><FONT size=2>Executing DBCC FREEPROCCACHE to kick ALL plans out of cache (fyi &#8211; you can see what&#8217;s in cache by querying master.dbo.syscacheobects)<\/FONT><\/FONT><br \/>\n<LI><FONT face=Verdana><FONT size=2>DATA on which the procedure depends changing&nbsp;enough to cause the statistics to be invalidated and therefore sql server proactively invalidate the plan <\/FONT><\/FONT><br \/>\n<LI><FONT face=Verdana><FONT size=2>Executing an sp_recompile (on the object(s) on which the procedure depends. This last one is also nice and something I sometimes force (off hours when a lot of indexes are being rebuilt and\/or added). You can execute <STRONG>sp_recompile tname<\/STRONG> and it will cause all plans which access this table to be invalidated.<\/FONT><\/FONT><\/LI><\/UL><\/LI><\/UL><\/LI><\/UL><br \/>\n<DIV><FONT face=Verdana size=3>Stored Procedures for Security:<\/FONT><\/DIV><br \/>\n<UL><br \/>\n<LI><FONT face=Verdana><FONT size=2>Stored Procedures can be secure AND easier to manage &#8211; in terms of permissions.<\/FONT> <\/FONT><br \/>\n<LI><FONT face=Verdana size=2>If I am the owner of a table and I create a procedure based on my table &#8211; I can give people access to the procedure withOUT giving base table permissions. This will allow better security and minimize the accidental &#8220;oh darns&#8221; when someone forgets a where clause on a delete. Imagine a user who types in DELETE Sales WHERE invnum = 1234 BUT highlights ONLY the DELETE Sales&#8230;whose problem is that? <STRONG>Mine<\/STRONG> as the DBA. We can all argue that users should not have ad-hoc access to the server (and I agree) BUT stored procedures create a layer of abstraction which is ALWAYS good especially if you are using DYNAMIC SQL. <\/FONT><\/LI><\/UL><br \/>\n<BLOCKQUOTE dir=ltr style=\"MARGIN-RIGHT: 0px\"><br \/>\n<BLOCKQUOTE dir=ltr style=\"MARGIN-RIGHT: 0px\"><br \/>\n<DIV><FONT face=Verdana size=2>Dynamic sql has many connotations&#8230; There is &#8220;Dynamic String Execution&#8221; which can be WITHIN a stored procedure and there&#8217;s Dynamic SQL which is what an application builds and sends off to the server. As for a client who sends an ad-hoc string to the server or for a dynamically built string within a stored procedure &#8211; BOTH REQUIRE that the user have the ability to execute the command directly &#8211; which means there&#8217;s more room for error. However, if it doesn&#8217;t need to be in a dynamic string (and many statements can be parameterized) then it could be in a stored procedure WITHOUT dynamic string exection withOUT the possible room for HUGE SQL injection problems\/errors. Speaking of SQL Injection &#8211; IF you use stored procedures and mostly only allow for Identifiers as parameters (i.e. tablename, viewname column names, etc.) then you can use the QUOTENAME() function to significantly reduce SQL injection.<\/FONT><\/DIV><\/BLOCKQUOTE><\/BLOCKQUOTE><br \/>\n<DIV><FONT face=Verdana size=3>Caching (in General) for better Performance:<\/FONT><\/DIV><br \/>\n<DIV><FONT face=Verdana><\/FONT>&nbsp;<\/DIV><br \/>\n<DIV><FONT face=Verdana>There are really three areas that need to be understood to really get the issues related to stored procedures:<\/FONT><\/DIV><br \/>\n<UL><br \/>\n<LI><FONT face=Verdana size=2>Ad-hoc statement caching (that was new in SQL Server 7.0 and higher)<\/FONT><br \/>\n<LI><FONT face=Verdana size=2>Forced statement caching (through sp_executesql)<\/FONT><br \/>\n<LI><FONT face=Verdana size=2>Stored procedure caching (by creating stored procedures)<\/FONT><\/LI><\/UL><br \/>\n<DIV><FONT face=Verdana><STRONG>Ad-hoc Statement Caching<\/STRONG><\/FONT><\/DIV><br \/>\n<DIV><FONT face=Verdana size=2>When a statement is deemed &#8220;safe&#8221; sql server will take EVEN ad-hoc statements and store a plan of execution in cache for subsequent users to use. For the plan to be re-used the statement has to be almost an identical match, the parameter has to be the EXACT same data type, the objects cannot be ambiguous and MOST statements won&#8217;t really benefit from this. If you want to see an example of ad-hoc statement caching do the following:<\/FONT><\/DIV><br \/>\n<BLOCKQUOTE dir=ltr style=\"MARGIN-RIGHT: 0px\"><br \/>\n<DIV><FONT face=Verdana size=2>&#8212; 1) Clear Cache with<BR><\/FONT><\/DIV><br \/>\n<DIV><FONT face=\"Lucida Console\" size=2>DBCC FREEPROCCACHE<\/FONT><\/DIV><FONT face=Verdana size=2><br \/>\n<DIV><BR>&#8212; 2) Look at what executable plans are in cache<BR><\/DIV><br \/>\n<DIV><FONT face=\"Lucida Console\">SELECT sc.* <BR>FROM master.dbo.syscacheobjects AS sc <BR>WHERE sc.cacheobjtype = &#8216;Executable Plan&#8217;<\/FONT><\/DIV><br \/>\n<DIV><BR>&#8212; 3) execute the following statement<BR><\/DIV><br \/>\n<DIV><FONT face=\"Lucida Console\">SELECT t.* <BR>FROM pubs.dbo.titles AS t <BR>WHERE t.price = 19.99<\/FONT><\/DIV><br \/>\n<DIV><BR>&#8212; 4) Look again at what executable plans are in cache and you&#8217;ll find that there&#8217;s a <\/DIV><br \/>\n<DIV>&#8212;&nbsp;&nbsp;&nbsp;&nbsp; plan for a NUMERIC(4,2) (look at the &#8220;sql&#8221; column in output &#8211; far RIGHT)<BR><BR><FONT face=\"Lucida Console\"><br \/>\n<DIV><FONT face=\"Lucida Console\">SELECT sc.* <BR>FROM master.dbo.syscacheobjects AS sc <BR>WHERE sc.cacheobjtype = &#8216;Executable Plan&#8217;<\/FONT><\/DIV><br \/>\n<DIV>&nbsp;<\/DIV><\/FONT><\/DIV><br \/>\n<DIV>&#8212; 5) If you execute the EXACT same statement with a 4,2 then you will get THAT plan<BR>&#8212;&nbsp;&nbsp;&nbsp;&nbsp; but if you execute with a 5,2 you&#8217;ll get a new plan (the plan is not safe). Execute this:<\/DIV><br \/>\n<DIV><BR><FONT face=\"Lucida Console\">SELECT t.* <BR>FROM pubs.dbo.titles AS t<BR>WHERE price = 199.99<\/FONT><\/DIV><br \/>\n<DIV><BR>&#8212; 6) Look again at what executable plans are in cache&#8230; <\/DIV><br \/>\n<DIV><BR><FONT face=\"Lucida Console\">SELECT sc.* <BR>FROM master.dbo.syscacheobjects AS sc<BR>WHERE sc.cacheobjtype = &#8216;Executable Plan&#8217;<\/FONT><\/DIV><\/BLOCKQUOTE><br \/>\n<DIV dir=ltr><\/FONT><FONT size=2><FONT face=Verdana>OK &#8211; so the fact that SQL Server can cache the plan is good&#8230; How often is something actually deemed safe &#8211; well, it&#8217;s not that likely. A better way of getting better plan re-use IS either of the two following BUT in these two cases it&#8217;s ALWAYS cached &#8211; which might not ALWAYS be good&#8230;. <\/FONT><\/DIV><\/FONT><br \/>\n<DIV><FONT face=Verdana size=2><\/FONT>&nbsp;<\/DIV><br \/>\n<DIV><FONT face=Verdana size=2><STRONG>Forced Statement Caching (through sp_executesql)<\/STRONG><\/FONT><\/DIV><br \/>\n<DIV><FONT face=Verdana size=2>This is good IF you know the plans are consistent (<EM>more on this coming up<\/EM>) and IF you want to type the data more strictly. Remember how SQL Server had to type the data above (numeric(4,2) or numeric(5,2), etc.). Well here you can type the parameters and force the plan to be cached. All subsequent uses will get the same plan:<\/FONT><\/DIV><br \/>\n<BLOCKQUOTE dir=ltr style=\"MARGIN-RIGHT: 0px\"><br \/>\n<DIV><FONT face=\"Lucida Console\" size=2>DECLARE @ExecStr&nbsp;&nbsp;&nbsp; nvarchar(4000)<BR>SELECT @ExecStr = N&#8217;SELECT t.* FROM dbo.titles AS t WHERE t.price = @price&#8217;<BR>EXEC sp_executesql @ExecStr,&nbsp;N&#8217;@price money&#8217;,&nbsp;19.99<\/FONT><\/DIV><\/BLOCKQUOTE><br \/>\n<DIV><FONT face=Verdana size=2>BUT this falls into the category of being hard to know and there are better ways with stored procedures. Generally, I recommend stored procedures over statement caching&#8230;<\/FONT><\/DIV><br \/>\n<DIV><FONT face=Verdana size=2><\/FONT>&nbsp;<\/DIV><br \/>\n<DIV><FONT face=Verdana size=2><STRONG>Stored procedure caching (by creating stored procedures)<\/STRONG><\/FONT><\/DIV><br \/>\n<DIV><FONT face=Verdana size=2>OK &#8211; I could go on for hours here and I&#8217;ll make it sufficient to say that NOT all procedures should be cached but MANY should. The ONLY way to get the security and consistency desired from stored procedures is to know when to save them and when not to. It also requires that you typically have more smaller stored procedures rather than fewer larger procedures (i.e. modular code is a KEY component to better performance). Anyway, having said this&#8230; IF the first person who executes the procedure (and there isn&#8217;t a plan for the procedure already in cache (<EM>and just to make this even more clear &#8211; stored procedure plans &#8211; when saved &#8211; are ONLY in cache they are NEVER saved to disk<\/EM>)) then a plan will be generated &#8211; and used for ALL subsequent users (unless a recompile is forced or occurs due to issues in third bullet in first section :). So now that there&#8217;s a plan &#8211; will that plan be perfect&#8230;not necessarily. I have a great script that shows this and it&#8217;s hard to explain over a short blog but the key point is this:&nbsp; (<EM>I took this small section from yet another of my emails so forgive the duplication<\/EM>): <\/FONT><\/DIV><br \/>\n<BLOCKQUOTE dir=ltr style=\"MARGIN-RIGHT: 0px\"><br \/>\n<DIV><FONT size=2><FONT face=Verdana>I would say that forced statement caching and stored procedure caching have the same problem(s) and that&#8217;s that NOT all statements should be cached. There are certain strategies that can help to minimize costly overhead and there are certainly tips\/tricks that we use to see if a plan should be saved or not. <FONT size=2>I think the number one thing I&#8217;d recommend is more modular procedures where the data returned SIGNIFICANTLY varies. The key reason is that smaller procedures tend to be more controllable. And &#8211; when a complex procedure is separately out then you can set that smaller more modular proc to be recompiled on each execution. That&#8217;s probably the easiest way. There are certainly other things you can do to more effectively determine the cause of the problem (and that would be better) but I&#8217;d say that forcing recompilation on a smaller piece of code that generates widely varying sizes\/sets of data is a good and generic solution.<\/FONT><\/FONT><\/FONT><\/DIV><\/BLOCKQUOTE><br \/>\n<DIV dir=ltr><FONT face=Verdana size=2>So &#8211; having gotten through all of this&#8230; <\/FONT><FONT face=Verdana size=2>If you really want to play with this A LOT more below are a couple of scripts that I think are VERY useful. I use these in my lectures on Optimizing Stored Procedure performance. Regardless, this is the correct order to read\/review\/learn these scripts:<\/FONT><\/DIV><br \/>\n<DIV dir=ltr><FONT face=Verdana size=2><\/FONT>&nbsp;<\/DIV><br \/>\n<DIV dir=ltr><FONT face=Verdana size=2>Create the Credit database (you can use <a href=\"https:\/\/www.sqlskills.com\/blogs\/kimberly\/content\/binary\/creditsampledb.zip\">CreditSampleDB.zip (55.79 KB)<\/a>&nbsp;to create it)<\/FONT><\/DIV><br \/>\n<DIV dir=ltr><FONT face=Verdana size=2>Use &#8220;<a href=\"http:\/\/staff.newtelligence.net\/playground\/clemensv\/kim\/content\/binary\/RecompilationIssues.sql\" class=\"broken_link\">RecompilationIssues.sql (3.67 KB)<\/a>&#8221; to get insight into bad plans being created\/saved\/re-used<\/FONT><\/DIV><br \/>\n<DIV dir=ltr><FONT face=Verdana size=2>Use &#8220;<a href=\"http:\/\/staff.newtelligence.net\/playground\/clemensv\/kim\/content\/binary\/ModularProcedures.SQL\" class=\"broken_link\">ModularProcedures.SQL (4.28 KB)<\/a>&#8221; to get insight into what happens even with procedures that use conditional logic &#8211; <EM>not that there&#8217;s anything wrong with that?!<\/EM><\/FONT><\/DIV><\/p>\n","protected":false},"excerpt":{"rendered":"<p>After a group of RDs started chatting about stored procedures I finally felt like I could join in. Often they talk about .Net and client stuff and well, quite honestly, I don&#8217;t care what you do with the data&#8230;only that I serve it up quickly. Ok, I&#8217;m really kidding here but my primary focus is [&hellip;]<\/p>\n","protected":false},"author":7,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[50,59],"tags":[],"class_list":["post-737","post","type-post","status-publish","format-standard","hentry","category-optimizing-procedural-code","category-security"],"_links":{"self":[{"href":"https:\/\/www.sqlskills.com\/blogs\/kimberly\/wp-json\/wp\/v2\/posts\/737","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.sqlskills.com\/blogs\/kimberly\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.sqlskills.com\/blogs\/kimberly\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.sqlskills.com\/blogs\/kimberly\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/www.sqlskills.com\/blogs\/kimberly\/wp-json\/wp\/v2\/comments?post=737"}],"version-history":[{"count":0,"href":"https:\/\/www.sqlskills.com\/blogs\/kimberly\/wp-json\/wp\/v2\/posts\/737\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.sqlskills.com\/blogs\/kimberly\/wp-json\/wp\/v2\/media?parent=737"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.sqlskills.com\/blogs\/kimberly\/wp-json\/wp\/v2\/categories?post=737"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.sqlskills.com\/blogs\/kimberly\/wp-json\/wp\/v2\/tags?post=737"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}