Does everybody get that? (Type system compatibility mode in drivers)

This post is an example of the "bring your own problem" approach in the classroom, inspired by upcoming SQLskills Immersion Events in August. See previous posting for the reason behind the blog post title.

The SQL Server "MAX" data types (VARCHAR(MAX), NVARCHAR(MAX), VARBINARY(MAX)) have been around for some time (SQL Server 2005) now. And the data types that they are meant to replace (TEXT, NTEXT, IMAGE) have been deprecated for just as long. The old data types somehow made their way in SQL Azure (sorry, Windows Azure SQL Database now), I was kind of shocked about that. It's a best practice to use the new data types.

A student in one of my classes once reported, at the "best practice" point in the proceedings, that changing the database from IMAGE to VARBINARY(MAX) had broken an old application. They were (and wanted to continue to) use the new ODBC driver. Is there any way to use the new data types with old apps? He wasn't using the obvious culprits (READTEXT and friends) in the application.

The problem ended up being with the application's use of column metedata. The application was using that metadata to allocate a buffer based on the maximum size of the data type. IMAGE had reported this maximum size as 2GB, VARBINARY(MAX) was reporting -1. The reason for the -1 was because, in SQL Server 2008, VARBINARY(MAX) column with the FILESTREAM attribute can be larger than the usual 2GB limit. It actually reports -1 in drivers dating back to SQL Server 2005, guess they were planning for the future then.

Fortunately, there is a way to have the driver use the SQL Server 2000 type system and return 2GB as maximum size for the MAX data types. In ODBC, this setting is a connection string parameter "DataTypeCompatibility=80", in ADO.NET it's "Type System Version=SQL Server 2000". In answering a question on a mail list recently, I was surprised to find out that the latest JDBC driver does appear to lack such a setting.

Does everybody get that?

Looking forward to seeing you in August…

@bobbeauch

Does everybody get that? (SQLCLR)

Just came back from TechEd, where I had a bit of time to think about training vs speaking. When I'm teaching, I'm not just speaking facts and doing neat demos (ta da!) at folks. I look at their eyes to make sure they "get it" and aren't satisfied until they do. During the presentation, I'd noticed that the lights were turned down low, and I couldn't do that. This manifested itself in my saying "Does everybody get that?". Maybe once too often… In a classroom, you also have more time to talk to individuals about their specific issues. I've heard this called "bring your own problem".

Many of my blog posts come from thinking about and solving (even after the class) problems that are brought up during classes. So I thought it would be worthwhile to present a series of them (without identifying info, naturally) that I haven't posted about yet.

Here's one on SQLCLR assemblies. I'm working on a payroll system and want to expose a set of calculations in a SQLCLR assembly. But, if all assemblies are loaded into the same .NET appdomain, I won't be able to keep a programmer that "knows better" from invoking my methods through reflection, thereby circumventing security. Having a different appdomain (you get one per assembly owner per database) means I won't be able to call to the other appdomain (for users that need to call my methods from other appdomains) without using a remoting technology. This seems like a bit of overhead, to use web services to call one SQLCLR appdomain from another. Didn't there used to be an "EXECUTE" permission on assemblies? (BTW, that permission did exist in 2005 beta 1, but didn't do anything, it was removed pre-RTM).

Well, how about using SQL? When you expose an algorithm (say, the pension calculation algorithm) as a T-SQL function/procedure, rather than just having your function available in an assembly to call via other SQLCLR .NET code, security is enforced through SQL permissions, the same as with other SQL objects.

This brings up another misconception about SQLCLR (.NET assemblies in SQL Server) code. Programmers sometimes think there is a special pipeline when calling from a .NET client to a SQLCLR object (e.g. a stored procedure) and that the rules (like type usage) of .NET apply. The only way to call from a .NET client to a SQLCLR stored procedure is by using SQL; that is, ADO.NET SqlCommand objects. And of course, you must define "entry points" into your .NET code using SQL DDL, such as CREATE PROCEDURE foo… EXTERNAL NAME [AssemblyName].[ClassName].[MethodName]. There is no .NET client-to-.NET sproc pipeline.

Get that?

@bobbeauch

Immersion Events: IEDev, IE4, and similar (but not the same) topics

SQL Server Immersion Events are coming up in August. I deal with two different groups of people, developers and DBAs, on a variety of topics. When it comes to the specialized SQL Server topics (like SQLCLR, SQLXML, Service Broker, etc), I though that it would be best to address the seperate groups, each with a different focus. And I've been asked "what's the difference" in coverage? Both groups may have input into feature usage, so they both want to know WHAT it is they are dealing with for planning purposes. And both want to know "how do you make it run fast"? That's where the similarity ends.

Developers want the know how to program using a feature and learn about the use cases where it makes sense.  Does it have it's own non-SQL language (like .NET or XQuery)? And how do I program that, what is the new syntax, new DML statements? Where do I look if I don't get the "right answer"? When is it best to use this feature as opposed to rolling my own equivalent in SQL? Does an equivalent in SQL even exist? How is this feature exposed within SQL (i.e. full-text CONTAINS, etc) and do these predicates, TVFs, etc, work well with other SQL constructs? Are there special constructs in the APIs I use, like ODBC or ADO.NET?

DBAs want to know how a feature affects a running server, and special considerations for backup/restore/reorganization and capacity planning. They look at a feature and think, how much disk space does this use and how much memory? What does the DDL look like? Are there new database objects, especially indexes? How about DMVs, metadata, trace events, and performance monitor counters to troubleshoot when things go wrong? What are the security repercussions?

That's why I've divided the topics more-or-less in two for the Immersion Events. IEDev is all about development, where IE4 covers some of the same topics for the more administratively inclined. Each one contains additional topics of interest to the corresponding group.

Hope this clears things up…

@bobbeauch

Using PowerShell with DAC 3.0

At my DAC talk at TechEd last week, the thing that seemed to cause the most interest was the PowerShell scripts to use DACPACs/BACPACs. I thought I'd post that here, along a question about cmdlets in general. Using PowerShell against the library allows admins to use all of the functionality that's in SSMS, with the options available in SqlPackage (think DacOptions object) and more (like BACPACs)… The premise is that administrators (DBAs or others) might want to integrate this functionality as part of their deployment into a PowerShell script.

PowerShell doesn't officially support .NET 4.0 assemblies until PowerShell 3.0, so you'll need a config file. You'd name this PowerShell.exe.config and place it in the same directory as the PowerShell.exe:

<?xml version="1.0"?>
<configuration>
  <startup useLegacyV2RuntimeActivationPolicy="true">
    <supportedRuntime version="v4.0.30319"/>
    <supportedRuntime version="v2.0.50727"/>
  </startup>
</configuration>

Then the scripts. Granted, I'm using the defaults, not the DacOptions or list of tables to export.

# load in DAC DLL (requires config file to support .NET 4.0)
# change file location for a 32-bit OS
add-type -path "C:\Program Files (x86)\Microsoft SQL Server\110\DAC\bin\Microsoft.SqlServer.Dac.dll"

# make DacServices object, needs a connection string
$d = new-object Microsoft.SqlServer.Dac.DacServices "server=."

# register events, if you want 'em
register-objectevent -in $d -eventname Message -source "msg" -action { out-host -in $Event.SourceArgs[1].Message.Message }

# Extract DACPAC from database
# Extract pubs database to a file using DAC application name pubs, version 1.2.3.4
$d.extract("c:\temp\pubs.dacpac", "pubs", "pubs", "1.2.3.4")

# Export schema and data from database pubsdac
$d.exportbacpac("c:\temp\pubsdac.bacpac", "pubsdac")

# Load dacpac from file & deploy to database named pubsnew
$dp = [Microsoft.SqlServer.Dac.DacPackage]::Load("c:\temp\pubs.dacpac")
$d.deploy($dp, "pubsnew")

# Load bacpac from file & import to database named pubsdac2
$bp = [Microsoft.SqlServer.Dac.BacPackage]::Load("c:\temp\pubsdac.bacpac")
$d.importbacpac($bp, "pubsdac2")

# clean up event
unregister-event -source "msg"


Now, the question for the reader. There has been a lot of work going on to make sets of APIs into libraries of cmdlets. I thought about this, but it seemed to be such a simple API that I'd be writing a cmdlet encapsulating 2-3 lines of code each time. So when is it useful to "cmdlet-ize" APIs? Always? Or is it too much to cmdlet-ize for every two lines of code? How about 5 lines of code? Is there a cutoff? Let me know on Twitter or in comments…

Thanks to those of you who attended the session.

@bobbeauch

I’ve got the last talk at TechEd and its almost LAST CALL

I've been at TechEd North America in Orlando this week and have seen some amazing new things. Upcoming operating systems, new hardware, and a detailed technical followup on the Azure (cloud) announcements from last week, including some Windows Azure SQL Database (was SQL Azure) upcoming enhancements. I've spend quite a bit of time at the Technical Learning Center and Ask the Experts Community Meeting answering questions about not only performance (one of the TLC themes), but also spatial data and indexing, full-text indexes, security (auditing), SQL Server and PowerShell, deploying with data-tier application packaging, and even a collation question. Popped in on the hands on labs, and visited a few vendors, especially if old friends were working there. It was a busy week.

And my old phone broke. I think its trying to tell me something… ;-)

Things aren't quite over yet. I've got the very last breakout session this afternoon, from 4:30-5:45. That would be DBA411 on integrating filetables and full-text search, including property search, and semantic search, and some other interesting new FTS features (time permitting). Last Call. The "last call" session (especially because its right before the attendee party) can be a bit lonely sometimes, as everyone is either rushing home or back to the hotels to get ready. An interesting thing, though, is that you tend to get the hard-core folks, the people that the REALLY interested in a particular feature. So if you come and keep me company at last call, perhaps I'll have time for some really hard-core info on the topic too.

As always, I'll be posting the demos from my two sessions on the SQLskills website next week. And, because there was so much interest in combining DAC and PowerShell writing a followup blog entry on that as well.

It was great to meet up with old friends and make new ones. And if you didn't come over from Europe for TechEd North America (I ran into a lot of Europeans here), I'll see you at TechEd Europe in Amsterdam in a few weeks! Cheers!

 @bobbeauch

SQL Server and Pooled vs. Non-pooled connections

A variation of this question came up a few weeks ago on a mail list that I subscribe to and it seems to come up a lot. "Can SQL Server distinguish a which connections use connection pooling?" The answer depends on how you ask the question, and I've tried to ask it in such a way that it's as ambiguous as it usually sounds. Here's what's happening.

Connection pooling is implemented almost entirely in the client stack. I'll refer to ADO.NET here (because it has some nice, client-side, perfmon counters) but this is true of OLE DB, ODBC, and JDBC as well, although ADO.NET uses different code. You can see the code with .NET reflector and trace it using the client-side trace. Connection pooling is implemented on a per-client, per-appdomain basis in ADO.NET, and all the connection string parameters must match for a pooled connection to be reused. In ADO.NET, connection pooling is on by default, unless you turn if off in the connection string.

When connection pooling opens a connection for the first time, it actually goes through the TDS message exchange to log in to the server. When the client-side code requests a connection close (or even connection dispose), the connection is not removed from the pool, log out does not occur on the server. Instead, you can think of the connection being "active" on the server, but not in use in the client's pool. When the client reuses the connection again, instead of logging on, the first batch of sQL executed causes a RESETCONNECTION bit to be set on the first TDS packet to the sent (there is also a related RESETCONNECTIONSKIPTRANSACTION bit, but that's not relevent for this discussion). This causes SQL Server to run the system stored procedure sp_reset_connection on the server's session. Running sp_reset_connection is a lot faster than redoing the entire login protocol exchange, that's the time-savings of connection pooling. If you consider sp_reset_connection as part of "connection pooling" (it's certainly essential for it to work properly), you can consider (maybe, during an argument) that part of "connection pooling" to be implemented in the server. I guess. But the client implements the pools, recycles the connections, and sends the reset bit for the server to act on.

Folks that think SQL Server can distinguish which connections use pooling in the client usually point to a the fact that the AuditLogin/AuditLogoff events in profiler indicate in the Event Subclass field whether or not a connection is pooled. And indeed they do. BTW, there is analogous information in XEvents.

So I set up a short test program that does the following 10 times (remember, connection pooling is on by default).This was run on .NET 4.0 client and SQL Server 2012 server:
  Create a new SqlConnection and SqlCommand
  Set the connection string
  Open the connection
  Execute the command
  Call close (and dispose for good measure) on the connection

The first time through, on the initial connnection to the server, Profiler reports a Non-pooled connection. Each subsequent time through, when the command is issued (remember, RESETCONNECTION is a bit on the first message in the batch, not part of the login message), Profiler reports the following three messages, followed by the SQL command that you actually execute:
  Logout (of a pooled connection)
  Call to sp_reset_connection
  Login (of a pooled connection)

So profiler reports a pooled logout/login (on a pooled connection) when we don't have a pooled connection (according to profiler) that can be logged out. A couple of other nit-picky observations: 
  If I execute the command the first time but only open and close/dispose the connection the 2-10th times (i.e. don't execute the command), no Logout/Login is reported.
  If I don't execute the command the first time (only open and close/dispose), but do execute the command on the second pass, logout/sp_reset_connection/login still happens.

There are also some perfmon counters that show what's going on. These are:
  .NET Data Provider for SqlClient (client connection pool information, per process)
  SQL Server:General statistics (counters are logins/sec, logouts/sec, connection resets/sec, logical connections)

When I look at these counters while running the test, they reflect what you expect. On the first time around the loop (the "real" login), I get 1 pooled connection on the client counters, although that's the one that profiler reported as Unpooled. There is 1 login as far as the server counters are concerned, and 9 connection resets, and 1 logout (at program termination). There is never more than 1 additional logical connection produced by the program.

So, back to our original question, "Can SQL Server distinguish a which connections use connection pooling". If I HAD to answer yes or no, I'd say NO, not until the client sends the RESETCONNECTION bit. Once a connection is reset, SQL Server could *surmise* that a connection was using pooling, but not until. I haven't located a place that the server keeps such information on a per-connection basis, although sys.dm_exec_sessions, sysprocesses, and sp_who, have a session state of "Dormant" when sp_reset_connection is being/has been run.

But, if the information in Profiler and XEvents Login/Logout useful? I'd say YES, it helps distinguish "real" logins/logouts from resets. Thanks to Erland Sommarskog for his valuable input in the original discussion. Hope this was helpful.

@bobbeauch
  

DACFx 3.0 – Import and Export with BACPACs

The last piece of the DAC puzzle (at least for now) is import and export. Export makes a non-transactionally consistant copy of database content that uses the BACPAC format (schema and data) and Import creates a new database with the BACPAC schema and data. Currently, its the most-used method of backing up and restoring a SQL Azure database. To make a transactionally-consistant backup of a SQL Azure database without using third-party tools, you must use "CREATE DATABASE … AS COPY OF" to create a transactionally-consistant copy of a database, then export the copy to a BACPAC.

Import and export can be accomplished by using the API directly. In DACFx 3.0, the API contains three overloads of the ExportBacpac method in the DacServices class, and an ImportBacpac class. If you don't want to program it yourself, Import and Export functionality is provided in SSMS 2012. There is also an Import/Export Web Service, but this currently differs from the SSMS 2012 Import/Export. The SQL Server Data Tools (SSDT) and SqlPackage.exe don't deal with BACPACs; they emit and consume only schema information via the DACPAC format.

During the development of SQL Server 2012, SSMS supported DAC 2.0. This used the data format for the BACPAC (compressed JSON format) along with the schema files from the 2.0 DACPAC, an extension of the 1.0/1.1 DACPAC. Just as with the DACPAC, DACFx 3.0 can consume 2.0 BACPACs, but DAC 2.0 cannot consume 3.0 BACPACs. No production version of SSMS and VS2010 projects supports DAC 2.0.

The Import/Export Web Service is a REST-base web service that runs at every Windows Azure data center and uses the BACPAC format. It currently (as of 5/22/2012) uses the BACPAC 2.0 format, not the DACFx 3.0 BACPAC format. The web service requires that your BACPAC input/output be network addressable; therefore is uses Windows Azure. You can use the web service directly from the Windows Azure portal; it's invocable using the Import and Export controls at the top of the page when a SQL Azure database is selected on the portal. There is also a web service client in the SQL DAC Examples project available on CodePlex.

The SQL DAC examples client provides one additional feature that's not available on the portal; you can create a BACPAC file that contains only a subset of tables. This is called "selective export" This functionality is also available when using SSMS 2012 export and the DACFx 3.0 API.

Because the web service uses a pre-DACFx 3.0 format, you'll get the following error if you try to import a DACFx 3.0 BACPAC using the web service "Unsupported BACPAC version.The BACPAC was created with a different version of the DAC Framework which will be supported in an upcoming service update". This happens whether you use the Windows Azure portal or the SQL DAC Examples project. You can, however, use to DACFx 3.0 BACPAC to populate a SQL Azure database by using SSMS 2012 Object Explorer, connecting to a SQL Azure database and using the "Import Data-Tier Application"  menu entry after right-clicking on the "Databases" folder.

When you use the web service, you're actually submitting a request to import or export using a BACPAC that the web service runs asynchronously. Submitting a request returns a request ID GUID. You can use this GUID along with the status command to see the results of your request using the "Status" function.

The BACPAC format does have a few limitations, even with DACFx 3.0. It doesn't currently support tables with sql_variant datatype columns, extended properties, or SQLCLR assemblies/SQLCLR database objects. Attempting to export these to a BACPAC produces a error. With sql_variant columns and extended properties, that's "selective export" comes in handy. SQL Azure doesn't support these items currently, you'll only see this error when doing a BACPAC export from SSMS 2012. The are some DacExtractOptions and DacDeployOptions that look like they'd be helpful when using the API, but there's currently no way to pass these in to the relevent Export and Import methods.

Chances are, you'll see some upgrades to the BACPAC functionality and tools in the upcoming months. I'll be updating as the story progresses. For now, the interesting thing about the BACPAC format is that, unlike SQL Server's native backup and restore format, it is backward as well as forward compatible for different versions of SQL Server, including SQL Azure, modulo feature support in different releases. Importing a BACPAC does take longer than restoring a backup, however. I found that out the easy way, when I received a "database backup" from SQL Azure as a BACPAC and was able to restore it to an on-premise SQL Server 2012 with a few clicks.

@bobbeauch

DACFx 3.0: The new file formats

In the last posting, I talked about the differences between DACFx 3.0 and previous versions from an API point of view. This time, I'll look at the files in a DACPAC and see how those differ.

DACPAC really stores a database model, that can be used to recreate the database and server (3.0 only) objects in the model, in an XML format. Note also that I've seen it written the a DACPAC is a simple zip file and you can unzip it to produce the same files that the "Unpack" Windows explorer shell extention (i.e. right-click on a DACPAC on a machine where DAC is installed and you'll see an "Unpack" context menu entry. This is not exactly true, as unzipping a V1.1 DACPAC on my system produces the following files:
 DacMetadata.xml
 LogicalObjectStream.xml
 PhysicalObjectStream.xml
 [Content Types].xml

And using the Unpack extention produces:
 DacMetadata.xml
 LogicalObjectStream.xml
 PhysicalObjectStream.xml
 model.sql

Note that the "unpacker" uses the object definitions in the XML files to generate a SQL script for the model. To stay on the V1.1 DACPAC for a minute, if the DACPAC has been produced using the VS2010 DAC Project, additional files may be present. These include:
 Script.PreDeployment.sql
 Script.PostDeployment.sql
 TargetSelection.xml

Pre and post deployment scripts are intuitively named. TargetSelection refers to the file the VS2010 DAC Project produces if you use a SQL target selection policy. With this project type, you can specify (editable using the ServerSelection.sqlpolicy node in the Solution Explorer for the project) which servers this DAC should be installed on. Choices include Collation, IsCaseSensitive, OSVersion, and more. Note: TargetSelection seems to be missing from the SSDT projects.

With an DACFx 3.0 DACPAC, different files appear. The "unpacker" will produce:
 DacMetadata.xml
 Origin.xml
 model.xml
 model.sql 

Note that, just with pre-3.0 versions, no .sql file appears in the unzipped version, so the .sql script is actually produced by the "unpacker". The model is strictly XML-based until unpacked. As interesting aside is that in DACFx 3.0, unpacking is supported by the API, using the Unpack method in the DacPackage or BacPackage classes. These methods support unpacking 3.0 or pre-3.0 DACPAC/BACPAC formats. The equivalent code in the pre-3.0 unpacker couldn't possibly know about future versions.

So what do these files contain?
 DacMetadata.xml – The name and version of the "application" that this DACPAC represents. Note that in 3.0, an XML namespace is added.
 Origin.xml (3.0 only) – Some metadata about where this DACPAC came from (which version, which tool, etc).

So the difference really amounts to the distinction between LogicalObjectStream.xml and PhysicalObjectStream.xml (pre-3.0) and model.xml (3.0). These files contain an XML model of a database. In appears that, in DACFx 3.0, the two XML files were consolidated into one. And all the additional database and server objects that DACFx 3.0 represents were added.

Finally, let's take a look at the model files. DACPAC pre-3.0 refers to two XML schema, in both the "Logical" and "Physical" model files. These are:
 http://schemas.microsoft.com/sqlserver/ManagementModel/Serialization/2010/11
 http://schemas.microsoft.com/sqlserver/RelationalEngine/Serialization/2010/11

I was not able to find a current version of the XSD files at the location implied by the namespace name (that's not a requirement of the XML schema spec, but sometimes it works), but I was able to find an older version of the ManagementModel schema at the schema.microsoft.com website.

In DACFx 3.0, there is reference to a single XML schema in the model.xml and DacMetadata.xml files:
 http://schemas.microsoft.com/sqlserver/dac/Serialization/2012/02

This schema appears at the location that matches its namespace name. And two additional XML schemas also exist related to DACFx 3.0. They are:
 http://schemas.microsoft.com/sqlserver/dac/DeployReport/2012/02
 http://schemas.microsoft.com/sqlserver/dac/DriftReport/2012/02

So, we've just taken a little spin through the items in a pre-3.0 and 3.0 DACPACs. The XML files and the information contained in those files have evolved greatly in DACFx 3.0. This is likely due to the integration of SQL Server Database Projects from VS2010 and the DAC Project in VS2010 SP1 into a single project type and format, that of SQL Server Data Tools, which subsumes both types of project. As for the concept of using DAC to manage "Deployment target" systems (i.e. the TargetSelection.xml file) depending on its utility and acceptence for use in database/application management, maybe this will return one day.

@bobbeauch

DACFx 3.0: The new programming API

Looking at the API and at the serialized form (i.e. the DACPAC), it turns out that DACFx 3.0 is not just "DAC V-next". It's an entire quantum change from all other versions, including DAC 2.0, the version that introduced the BACPAC (serialized schema and data). This has some interesting repercussions with compatibility. Let's start with the "native" API (meaning, programming over the DLL directly, I'll leave the REST API used by SQL Azure Portal's Import/Export for another time).

The DACFx 3.0 API lives in a different DLL than previous versions of DAC. The old DAC (2.0 and below) lives in Microsoft.SqlServer.Management.Dac; DACFx 3.0 lives in Microsoft.SqlServer.Dac. The API is quite different with respect to classes and methods then the one I wrote about in SQL Server Magazine's Jan 2012 issue. Instead of using the DacStore and DacExtractionUnit classes from previous versions, most everything has been refactored into the all-new DacServices controller class. This new controller class contains most of the methods from the original DAC (albeit with different names like Deploy, Extract, Register, Unregister) along with new methods that generate a drift report and methods to Export and Import the BACPAC (schema and data) format. The one method that's been dropped in the new API is the Uninstall (unregister and drop database) method.

One other difference is that DACFx 3.0 can work with DACPACs and BACPACs from previous DAC versions, but can only emit DAC 3.0 formats. The upshot of this is you can't use a DACPAC from SQL Server Data Tools or SSMS 2012 with earlier versions or tools.

The nice improvement we get from the DACFx 3.0 format is support for almost every object in SQL Server 2005-2012. In addition, when using the 3.0 APIs directly, or with SSDT, there are infinitely more options for working with DACPAC, encapsulated in the DacDeployOptions or DacExtractOptions classes. These options are exposed by SSDT as project properties, and its command-line utility, SqlPackage as command-line parameters as well. Some random examples would be the deploy option to "BlockWhenDriftDetected" or the extract option to "IgnoreUserLoginMappings".

You can read the documentation for the new classes in Microsoft.SqlServer.Dac here.

@bobbeauch

Scripting SQL Server databases sans SMO Scripter

I can script out a database schema (and optionally data) from SQL Server databases. Any version from 2005 and beyond, as well as SQL Azure Database. And most all instance level objects.  And I'm not the SMO scripter object. Or a DBA. Who am I?

I would be DACFx 3.0. And I can do some things (unlike SMO scripter, or a busy DBA) like script out deltas (schema only), given any two databases or DACPACs. Or any combination of SSDT project, database, and DACPAC. 

When DAC (that's data-tier applications, not dedicated administrator connection) was introduced back around SQL Server 2008 R2, there were a subset of "most likely candidate" database objects it could script. This increased over time to include all SQL Azure database objects. But in DACFx 3.0, which appears in SSMS 2012, SSDT (along with its command-line brother sqlpackage.exe), and soon the Azure portal, there is almost full-fidelity up to SQL Server 2012. There are still some rough edges being worked on (along with enhancements), both in the functionality and in the integration with SSMS, as you might expect.

SSMS and SQL Server's PowerShell provider are both built with SMO, and SMO's functionality roughly follows what SSMS supports (e.g. before SMO 2012 there was no SSMS/SMO support for extended events; now there is, in both places). SMO is much, much more granular in its API… think "almost every SSMS dialog is built over SMO". DACFx providers a discrete set of (less than 10) utility functions. But in SSMS 2008 R2, 2008 R2 SP1, and 2012, DACFx makes its presence felt. For example in SSMS 2012, we have, in SSMS Object Explorer:
 
Databases Tab:
    Deploy (from DACPAC)
    Import (from BACPAC)
Individual Databases/Tasks:
    Extract (to DACPAC)
    Deploy Database to Azure (Migrate via BACPAC)
    Export (to BACPAC)
    Register (to MSDB, to master on SQL Azure)
    Upgrade (existing registration and schema)
    Unregister

In SSMS, these functions are fairly rigid, i.e. there aren't many options, failure to comply with the "default options" can cause failure. But with SQLPackage.exe and SSDT and the DACFx API, a plethora of options exist (albeit not as many as SMO scripter, but some of scripter's options are for code formatting). These options live in the DacDeployOptions and DacExtractOptions classes in the API (that is Microsoft.SqlServer.Dac.dll) and are exposed as command-line switches in SQLPackage.exe or properties in the SSDT project properties page. And documented in the SQL Server documentation. 

If you don't want to install SSDT on your machine, you can download DACFx 3.0 as part of the SQL Server 2012 Feature Pack. It's called "Microsoft® SQL Server® 2012 Data-Tier Application Framework".

More to come…

@bobbeauch