The Rest of the Story: Co-existing VS2010 and SSDT database projects

After my last adventure, I thought it would be interesting to try the "upgrade path" with SSDT and older "Database Projects" (i.e. Visual Studio for Database Professionals, or whatever the last name was before we went to the nice SSDT acronym, I always called them "Data Dude" projects after Gert Drapers, the original Data Dude).

Note first-off that when you have both VS2010 and SSDT projects installed into Visual Studio, there are two main menus, one called "SQL" and the other called "Data". In case you can't guess, "Data" is the one that goes with the "older" projects (it contains an entry for "Data Compare", a feature that SSDT doesn't have yet). And "SQL" is the SSDT project menu. There's a few "dualing toolbars" too; luckily for me, I'm not a big toolbar user in VS.

At least in VS2010 the Database Projects were always version-specific in name so I know that I can't target SQL Server 2012 in a "SQL Server 2005 or 2008 database project". Easy. Opening one of these projects brings up a "Convert to SSDT" dialog. This works as advertised, just
realize that there are some "artifacts" that are not converted because the corresponding
feature doesn't exist in SSDT yet (e.g. Data Generation Plans). Backing up the old project is
a good idea in any case. See this page for a list.

Finally, there are Data Tier Application projects. These support SQL Azure and the subset of
database objects that SQL Azure supports. You can create a DAC Project in Visual Studio after
the SSDT tools are installed. Interestingly (but expected), you can't import database objects
from a SQL Server 2012 database, even if the database only contains the subset of objects
that DAC supported in DAC V2.0, you're presented with an "Unable to connect to database"
message. Surprisingly, if you set your SQL Server 2012 database to Version 90 or 100
compatibility mode (thats SQL2005 or SQL2008 compatibility), creating a DAC project and importing the
database now works! (if it contains the right subset of objects). And of course the project
type would work (at least at this point in time) with a SQL Azure database or importing a
V2.0 (but not V3.0) DACPAC.
Bringing up an existing VS DAC project brings the (at this point) familiar "Convert" dialog.
Bear in mind that DAC projects could have some of the same unsupported artifacts as Database
Projects. 

Got it? So now you can co-exist with both/all sets of SQL database projects, database-version
willing. There are some nice new features in SSDT that make it compelling to convert, and the
"catch up" features may be added soon, because SSDT will be updated "on a SQL Azure cadance".
That's once every 3-6 months. And, come Visual Studio 11 (available in CTP/beta now), there
will ONLY be SSDT projects. So…Enjoy!

@bobbeauch

Adventures in autodeploy: SQL Server 2012, SSDT, and SQLCLR

I'm the kind of person that doesn't often/always use Visual Studio "autodeploy" (that is, the "Deploy Solution" option) for SQLCLR projects. It's always been missing things (like ALTER ASSEMBLY) and never quite kept up with the newer things you could do with SQLCLR (e.g. ordered table-valued functions, multiparameter aggregates). But every once-in-a-while, especially while I'm teaching and writing example assemblies on-the-fly, its a useful way to get things cataloged fast without writing DDL.

So when I installed SQL Server 2012 and the SQL Server Data Tools on top of an existing VS2010 installation, I was not entirely surprised that, when opening a SQLCLR project, I was prompted to "convert to an SSDT project or leave it alone". Either choice promised me that "I wouldn't be prompted again" but I didn't want that. So I simply cancelled out of the dialog.

However, VS2010 SQLCLR projects autodeploy *doesn't work* against a SQL2012 database. Ever.
Even with the simplest projects. A profiler trace of the proceedings looks like some of the
DDL it generates is incompatible with SQL Server 2012. Hrmph.

So now I choose the "convert" option, and expect to see a full-fledged SSDT project in
Solution Explorer after the conversion. I don't. Looks like I have approximately the same
project. Not so. Looking at the project properties gives me all the SSDT properties, and
using "Add/New Item" on the project gives the entire list of (85 or so) items that you can
add to an SSDT project. So I DO have a full-fledged SSDT project. Does not mean I'll have to
import the rest of the database objects to get "autodeploy" (there's no "Deploy Solution" but
there is a Publish… option) to work??

Let's try this on the pubs database (I can always re-create the pubs database if something
goes wrong). And "Publish" simply publishes the assembly, leaving the rest of the database
undisturbed. Turns out that is what I want, so…great.

Now, I've made the moral equivalent of a SQL Server 2012/VS2012 SQLCLR project by conversion.
How do I make a new one? There no choice with SSDT but "Other Languages/SQL Server/SQL Server
Database Project". Made an empty one of those. So let's make an assembly. Add/New Item…and
the SQLCLR items don't appear in the list of all new items. But there is a branch off the
tree for "SQL CLR" and "SQL CLR C#". The "SQL CLR" is VB.NET. So first, I need to add an
assembly info file, if I want one of these. So far, so good, but there are no referenced
assemblies other than .NET 4.0's System.Dll (the properties window, SQLCLR tab, show Assembly
Info and fill in the dialog does the same thing). It looks like it's smart enough to build a .NET 4.0 target because my target database in project properties is SQL Server 2012. Now let's add a SQLCLR UDF (the default
template for this no longer returns "Hello", but String.Empty, hmmm…). And I do get
System.Data and System.Xml added to the references. And Build and Publish works. BTW, during
the "Publish" process, you also have the ability the generate a publishing script rather than
publish in real-time. That's nice.

So, I'll go through of compare the generated DDL later and see if I can do anything in SQLCLR
2012 that SQLCLR 2008 autodeploy (Publish/Deploy) can't do. For now, I at least have a way to
replicate the functionality of SQLCLR projects. That's works with a SQL Server 2012 (and 2008
and 2005, though I didn't try each one, I'll take them at their word, for now) database. Whew! But, once you've converted the
original VS2010 project (unless you choose, "save old project" during conversion) you can't
go "home" anymore… You're in SSDT-project-land now.

@bobbeauch

Demos from March-April European events posted

As I'm getting ready for my week-long SQLskills Immersion Event in Chicago next week, I got word that the demos from my events in Europe in late March/early April have been posted on the Resources part of the SQLskills website.

I got to catch up with a lot of old friends and make a lot of new ones at these events. Hope you liked the talks.

@bobbeauch

Thanks for a great time last week, demos coming

Thanks to all the folks who attended my sessions at DevWeek, SQLBits, and SQL Saturday Dublin. Hope that you enjoyed the sessions. It was really nice to meet up with old friends and also to make new ones.

As promised, the demos will be posted on the SQLskills website, but it may be a week or so, as I'm not headed home directly. It's on to Stockholm for a 2-day SQL Server 2012 for developers seminar before returning home at the end of the week. So it will be "soon".

Cheers, Bob

@bobbeauch

On the road…in a country near you?

Starting the middle of next week I'm about to embark on a series of speaking/teaching engagements in Europe. Although I've mentioned some of these before, and some of these are already sold out, it bears mentioning again. If you've read my "content" (books, blog posts, whitepapers, articles) in the past, and even if you haven't, if you're at these events, please stop by and say hi. Events like this are a good chance not only to catch up with old friends, but to make new ones. I'll be at:

SQL Saturday in Dublin – my first trip to Dublin and first SQL Saturday, really looking forward to it. Speaking on Searching files and data with SQL Server 2012 (a.k.a Filestream/FileTable and Full-text search improvements/internals).

DevWeek in London – A chance to catch up with old friends. I'm doing a precon here on SQL Server 2012 for Developers and a few talks; see the website for a schedule.

SQLBits X London – I've always wanted be at one of these. Speaking on Friday afternoon on SQL Server 2012 Spatial and Saturday on Extended Events – The New Trace.

Addskills Expertseminarium series – Two days of SQL Server 2012 and performance (Apr 3-4) for database developers. Addskills website has the outline. I always enjoy Stockholm and the Addskills folks are great hosts.

And, let's not forget the US and our very own SQLskills event. It's an almost sold-out SQLskills Immersion Event in Chicago in April. Five days of intense training on everything a database developer needs to know. And more.

Hope to see you at one of these events. Cheers!

@bobbeauch

Security Best Practices Whitepaper for SQL Server 2012 is Live.

I recently completed another update to the whitepaper, now entitled "SQL Server 2012 Security Best Practices – Operational and Administrative Tasks" to cover best usage of the security features in SQL Server 2012. And, concurrently with the launch of SQL Server 2012, it was available on the web a few days ago. Many thanks to the reviewers. Enjoy!

@bobbeauch

SQL Server 2012 at DevWeek London at the end of March

Hope that you'll find time in your schedule to attend DevWeek 2012 in London, March 26-30. I'll be doing a preconference talk, an entire day of SQL Server 2012 for Developers. In addition, I present some breakouts on SQL Server security best practices, Extended Events and the new profiler, Mission-critical SQLCLR and other relevent topics. Here's hoping to see you there, drop by and say hi!

@bobbeauch

New T-SQL 2012 scalar functions (all but 1) work in SQL Azure

Back when I was saw SQL Azure was producing an @@version string that mentioned version 11 (that's SQL Server 2012's major version number), I'd started looking at the version number every week or so. And looking for SQL Server 2012 functionality in SQL Azure. See the blog post "SQL Denali T-SQL features in SQL Azure now" for a list of the first functions supported. I'm not going to list them all again here.
And Ed Katibah's (a.k.a Spatial Ed) latest list of SQL Server 2012 spatial functionality in SQL Azure.

This week, I'd thought to look at the SQL Azure version again. I get:

Microsoft SQL Azure (RTM) – 11.0.1831.30
 Jan 27 2012 23:11:55
 Copyright (c) Microsoft Corporation

This is different (newer) than when I previously looked, so its time to look for new T-SQL 2012 functions again. And to find some. 13 of the 14 new T-SQL scalar functions (all of them except TRY_CONVERT, I wonder why that one didn't make it) are in. That would be: FORMAT, PARSE, TRY_PARSE, IIF, CHOOSE, CONCAT, EOMONTH, and the new "date-from-parts" series.

In addition, the T-SQL 2012 updated version of the T-SQL LOG function that allows a logarithm's base as a second argument works in SQL Azure as well.

There's also one more new function that isn't mentioned in the books online (I hit it by mistake when I thought I was somehow mistaken about there being a TRY_CONVERT and it works in SQL Server 2012 RC0 as well), that is TRY_CAST. It's analogous to TRY_CONVERT, as CAST is analogous to CONVERT, modulo CONVERT's expression and style arguments.

This isn't a complete implementation of SQL Server 2012's T-SQL improvements in SQL Azure, though. SQL Azure is still waiting for:

1. Sequences
2. UTF-16 collations with supplimentary characters
3. Windowing (OVER clause enhancements, LAG/LEAD and others, new analytical functions)

I didn't notice any change in spatial support moving more towards to SQL Server 2012 feature set this time, maybe Ed noticed something.

Well, a bit at a time, I guess. Welcome the new T-SQL functions to SQL Azure.

@bobbeauch

XEvents in SQL Server 2012: No more “lazy XML” in event harvesting scripts

I've said before how thrilled I am will the new extended event UI in SSMS for SQL Server 2012. However, you might be one of the early adopters who made up their own scripts to define extended event sessions, and use hardcoded scripts to harvest the results. So, you may run into this problem if you're using what I call "lazy XML" in the event harvesting script. Take, as an example, an extended event session defined as follows with 3 actions:

create event session errorsession on server
add event sqlserver.error_reported
(
action
(
package0.callstack,   
sqlserver.session_id,  
sqlserver.sql_text    
)
where error = 547 and package0.counter <= 3 
)
add target package0.ring_buffer
with (max_dispatch_latency = 1 seconds)
go

In previous versions, you could pretty much depend on the XML presenting the actions in order. So the following XML would return a subset of the event information in rows and columns.

SELECT
  Data2.Results.value ('(data/.)[1]', 'int') AS ErrorNumber,
  Data2.Results.value ('(action/.)[2]', 'int') AS Session,
  Data2.Results.value ('(action/.)[3]', 'nvarchar(max)') AS SQLStatement
from
(
select CAST(xet.target_data as xml) as data
from sys.dm_xe_session_targets xet
join sys.dm_xe_sessions xe
on (xe.address = xet.event_session_address)
where xe.name = 'errorsession') events
cross apply Data.nodes ('//RingBufferTarget/event')
  AS Data2 (Results)

The XML above is based on the assumption that action 2 is the sqlserver.session_id and action 3 is the sqlserver.sql_text, because it doesn't check names, just blindly uses the ordinal number in XPath. It makes the XML easier to write and a bit faster to execute, but its "lazy" XML. This order of actions was a valid assumption in SQL Server 2008; its not valid in SQL Server 2012. The data fragment containing the action data appears like this:

    <action name="sql_text" package="sqlserver">
      <type name="unicode_string" package="package0" />
      <value> 
        DELETE pubs.dbo.jobs
      </value>
    </action>
    <action name="session_id" package="sqlserver">
      <type name="uint16" package="package0" />
      <value>53</value>
    </action>
    <action name="callstack" package="package0">
      <type name="callstack" package="package0" />
      <value>..callstack elided…</value>
    </action>

So now, sql_text (the third action defined) is the first action presented. The fragile harvesting script will break. So, to be one the safer side, if you have any such scripts change them to actually look for the element for want using a named XPath predicate, like this:

SELECT
  Data2.Results.value ('(data[@name="error_number"]/.)[1]', 'int') AS ErrorNumber,
  Data2.Results.value ('(action[@name="session_id"]/.)[1]', 'int') AS Session,
  Data2.Results.value ('(action[@name="sql_text"]/.)[1]', 'nvarchar(max)') AS SQLStatement
from
(
select CAST(xet.target_data as xml) as data
from sys.dm_xe_session_targets xet
join sys.dm_xe_sessions xe
on (xe.address = xet.event_session_address)
where xe.name = 'errorsession') events
cross apply Data.nodes ('//RingBufferTarget/event')
  AS Data2 (Results);

Note that it's probably better to do this with the "data" items too (ErrorNumber in the query above), although you may be a little safer with these, as they have a defined schema per-event. But BOL does point out that events have a "versioned" schema. The actions can be defined in any order, so make your scripts more robust. Use XPath predicates. And if you're not on SQL Server 2012 yet, change your scripts now.

@bobbeauch

SQL Server 2012 FileTable and T-SQL on AlwaysOn secondary replicas

OK, back to PathName() with AlwaysOn, which I started on in the previous blog post. PathName() in SQL Server 2012 doesn't return the computer name by default ("AlwaysOn1" or "AlwaysOn2" in my example) but returns the VNN name (virtual network name). That is, it returns the availability group "listener share" name. In my case, the PathName() would start with \\AlwaysOnAG1. There's an additional option in SQL Server 2012 PathName(), that allows you to return the current replica name. So, when "AlwaysOn1" is the current primary replica, it returns that; when we fail over to "AlwaysOn2", that's what name is returned. In addition, GetFileNamespacePath() and FileTableRootPath() always return the availability group listener name, they don't have an option to return the current replica name.

Finally, I'll discuss using AlwaysOn, FilePath(), T-SQL access and secondary replicas.In theory, ADO.NET 4.02 (and, of course, SQLNativeClient 11ODBC/OLE DB and Microsoft JDBC 4.0 driver) allows you to declare your ApplicationIntent as ReadOnly or ReadWrite. ReadWrite is the default. Adding either ApplicationIntent=ReadOnly or ApplicationIntent=ReadWrite in the SSMS Connection dialog is accepted, but either one allows connection to the secondary replica. This is because, by default, secondary replicas are configured with "Readable Secondary=Yes" rather than "Readable Secondary=Read-intent". Configuring a secondary as "Readable secondary=Read-Intent" disallows connections that do not specify ApplicationIntent=ReadOnly in the connection string. Obviously, connecting as "ReadWrite" to a ReadOnly replica errors out if you try and do something other than read. Check out Sunil's excellent blog post on the motivations for these secondary replica settings.

Now that we're connected (in T-SQL) to the secondary replica, let's try and get the PathName() from here:

SELECT Name, file_stream.PathName() FROM NWTab;

Msg 33447, Level 16, State 1, Line 1
Cannot access file_stream column in FileTable 'NWTab', because FileTable doesn't support row versioning. Either set transaction level to something other than READ COMMITTED SNAPSHOT or SNAPSHOT, or use READCOMMITTEDLOCK table hint.

Now, this was surprising…I checked the snapshot info in sys.databases and also DBCC USEROPTIONS. Neither snapshot isolation level was turned on or being used. And setting the iso level to, say, repeatable read or even the dreaded read uncommitted didn't help. But using the READCOMMITTEDLOCK did. Hmmm…

But of course, Sunil had the answer for this one too…"To avoid this blocking completely, all transaction isolation levels used in an application on the secondary replica are mapped transparently to Snapshot Isolation…" And this behavior is more obliquely documented in the BOL, here. But Sunil's blog post does go on to say "You may ask how about locking hints? Well, all locking hints are ignored." But the READCOMMITTEDLOCK locking hint does work in this case (modulo the effects it might have on performance, as Sunil mentions). So at this point I wonder if the READCOMMITTEDLOCK locking hint on a FileTable has a special dispensation (other locking hints, do, in fact, fail to have the desired effect), but I'll need to revisit this in 2012 RTM. BTW, it's only using the file_stream column of the FileTable that causes this, other columns work fine. But PathName() and GetFileNamespacePath() are methods on the column, so you can't use these, or access the column in T-SQL, on readonly secondaries without the hint.

Back to BOL comment, changing the NON_TRANSACTED_ACCESS to READ_ONLY (on the primary replica) changes the behavior on the secondary as promised, but now I can't write to the FileTable using the file system. As expected.

Two more things to mention. Firstly, Msg 33447 isn't a mistaken error message. You can, of course, attempt to access the file_stream column in a FileTable from the *primary* replica using Snapshot isolation. Then you'll receive the error, and changing the iso level WILL help. When enabling read-committed snapshot or snapshot you DO get this useful (but slightly misleading, IMHO) message on the ALTER DATABASE DDL statement: "When the FILESTREAM database option NON_TRANSACTED_ACCESS is set to FULL and the READ_COMMITTED_SNAPSHOT or the ALLOW_SNAPSHOT_ISOLATION options are on, T-SQL and transactional read access to FILESTREAM data in the context of a FILETABLE is blocked.". That's not quite true; if the options are on and you *USE* snapshot iso levels, you'll get the error. Changing the iso level to a non-snapshot iso level or using the READCOMMITTEDLOCK hint WILL succeed on the primary.

Finally, remember that "ordinary" (i.e. non-FileTable) tables containing filestream columns DO support snapshot isolation levels as of SQL Server 2008 R2. It should be only the FileTable, with its non-transacted access, that has this behavior with snapshot isolation on the primary and readonly secondaries. And, in fact, this does test out to be the case, an "ordinary" filestream column works fine, even in the same database.

Enough? ;-) I even turned moderated blog comments back on (against my better judgement, the amount of spam comments is amazing), if this is a useful discussion vehicle. Or, you know where to find me… Cheers.

NB: Since writing this post, I discovered this post on Availability Group and Filetable – http://dangerousdba.blogspot.com/2012/07/filetable-with-alwayson-ags-bug.html. Thanks Rick!

@bobbeauch