Have you got something you've found out that you think others might be interested in. Then submit a session for the next SQLBits on 13th September 2008.

http://www.sqlbits.com/information/SessionSubmission.aspx

You need to be logged in to fill in a BIO and submit a session.

Once we have all the sessions in we will open voting to get the community to decide which sessions they would like to see.

If you're stuck for ideas have a look at the sessions from the previous events.

http://www.sqlbits.com/information/event2/PublicSessions.aspx

http://www.sqlbits.com/information/event1/PublicSessions.aspx

 


This is cross posted from my SQLBlogcasts blog which can be found here, http://sqlblogcasts.com/blogs/simons/SQLBits III - Submit your sessions now

Categories:


The MERGE statement is a new statement introduced in SQL Server 2008 that allows you to perform inserts, updates and deletes in one statement. You use a rowset as the source for the MERGE and as with all rowsets this can have any number of rows in it. For an introduction see “MERGE an introduction”.

The challenge with deletes is that on the client the row has been deleted, how do you handle the fact you no longer have the row when you use MERGE.

Assuming you are passing a set of data to your MERGE statement there are two way of handling deletes.

The first is the classic example that you will see in most examples. In this solution you work on the principle that if the row doesn’t exist in the source rowset then it must have been deleted and so you perform a DELETE.

This is done by using the WHEN TARGET NOT MATCHED clause of the MERGE statement

DECLARE @ChangedOrderDetail SalesOrderDetail_Type

DECLARE @SalesOrderID int

 

MERGE Sales.SalesOrderDetail OrderDetail

USING @ChangedOrderDetail    Chng        ON Chng.[SalesOrderDetailID] = OrderDetail.[SalesOrderDetailID]

                                        AND Chng.[SalesOrderID]       = OrderDetail.[SalesOrderID]

WHEN MATCHED THEN

    UPDATE SET [SalesOrderID]            = Chng.SalesOrderID           

              ,[CarrierTrackingNumber]   = Chng.CarrierTrackingNumber 

              ,[OrderQty]                = Chng.OrderQty               

              ,[ProductID]               = Chng.ProductID               

              ,[SpecialOfferID]          = Chng.SpecialOfferID           

              ,[UnitPrice]               = Chng.UnitPrice               

              ,[UnitPriceDiscount]       = Chng.UnitPriceDiscount       

              ,[rowguid]                 = Chng.rowguid               

              ,[ModifiedDate]            = Chng.ModifiedDate           

WHEN NOT MATCHED THEN

   INSERT ([SalesOrderID],[CarrierTrackingNumber],[OrderQty],[ProductID],[SpecialOfferID],[UnitPrice]

          ,[UnitPriceDiscount],[rowguid],[ModifiedDate])

   VALUES (Chng.SalesOrderID,Chng.CarrierTrackingNumber,Chng.OrderQty,Chng.ProductID

          ,Chng.SpecialOfferID ,Chng.UnitPrice,Chng.UnitPriceDiscount,Chng.rowguid,Chng.ModifiedDate);

WHEN NOT MATCHED IN SOURCE

   DELETE

What this does is perform a scan of the target table  (OrderLines) looking for rows in the source data (@ChangedOrderLines). You will notice I said scan and thats what it has to do, it has to check every row in the target table against the source data. This is a very expensive operation as most scans are. Furthermore it only works if the source data should represent ALL the rows in the target table.

Imagine you have an OrderLines table with 1 million rows. You are not going to retrieve all 1 million rows from your client application to update and then send back to the database. Instead you will return all the OrderLines for a specific Order.

If you use the DELETE pattern above, what will happen is that, your source data will only have data for one Order and won’t have any data for ALL the other Orders in the system. This will mean that ALL the OrderLines for the other orders will NOT MATCH IN TARGET and so be deleted.

This is obviously not ideal.

So whats the solution?

The solution is to use soft deletes. This means that you don’t DELETE the record but instead flag the record as deleted. We can then use the additional feature of MERGE which allows a condition to be added to the WHEN clause (a SQL Server specific extension not in the standard).

This turns our MERGE statement into the following.

 

DECLARE @ChangedOrderDetail SalesOrderDetail_Type

DECLARE @SalesOrderID int

 

MERGE Sales.SalesOrderDetail OrderDetail

USING @ChangedOrderDetail    Chng        ON Chng.[SalesOrderDetailID] = OrderDetail.[SalesOrderDetailID]

                                        AND Chng.[SalesOrderID]       = OrderDetail.[SalesOrderID]

--Note DELETEs are now done here

WHEN MATCHED AND Chng.Action = 'D' THEN

    DELETE

----

WHEN MATCHED THEN

    UPDATE SET [SalesOrderID]            = Chng.SalesOrderID           

              ,[CarrierTrackingNumber]   = Chng.CarrierTrackingNumber 

              ,[OrderQty]                = Chng.OrderQty               

              ,[ProductID]               = Chng.ProductID               

              ,[SpecialOfferID]          = Chng.SpecialOfferID           

              ,[UnitPrice]               = Chng.UnitPrice               

              ,[UnitPriceDiscount]       = Chng.UnitPriceDiscount       

              ,[rowguid]                 = Chng.rowguid               

              ,[ModifiedDate]            = Chng.ModifiedDate           

WHEN NOT MATCHED THEN

   INSERT ([SalesOrderID],[CarrierTrackingNumber],[OrderQty],[ProductID],[SpecialOfferID],[UnitPrice]

          ,[UnitPriceDiscount],[rowguid],[ModifiedDate])

   VALUES (Chng.SalesOrderID,Chng.CarrierTrackingNumber,Chng.OrderQty,Chng.ProductID

          ,Chng.SpecialOfferID ,Chng.UnitPrice,Chng.UnitPriceDiscount,Chng.rowguid,Chng.ModifiedDate);

 

This results in a much better query plan as the query engine only as to find those OrderLines that match those in the @ChangedOrderLines table parameter.

Using this pattern does require you to do some additional work on your client, its not difficult and I will show you how you can do that in the next article.

NOTE: This examples use the new syntax for MERGE introduced in RC0 of SQL SERVER 2008.

 


This is cross posted from my SQLBlogcasts blog which can be found here, http://sqlblogcasts.com/blogs/simons/SQL Server 2008 - Handling deletes with MERGE

Categories:


The MERGE statement and Table Valued Parameters (TVPs) are two new features in SQL Server 2008 that make the process of doing DML (Data Manipulation i.e. INSERT, UPDATE and DELETE) on multiple rows very easy. This article shows how you can adapt

Historically to save data using the DataAdpater you configure it with insert, update and delete statements, pass it a DataTable of changed data and it will then look at the DataTable and fire off the relevant insert, update or delete statement.

This is fine except it fires off a statement for each change. This isn’t very efficient due to, overheads with the network, making a connection to the database and performing a single row change in the database. We ideally want to reduce network calls and perform SET based operations in the database. TVPs and the MERGE statement allow us to do just that.

So how does the DataAdapater work using a TVP. The short answer is, it doesn’t. Fortunately its fairly easy to amend your code to use TVPs

If you’ve read the article on handling DELETEs with MERGE you will know that the best way to handle DELETES with MERGE is to pass them as soft deletes, i.e. flag the row as DELETED using an extra column in the table.

To do that we take the DataTable containing our changes and we add a column. We now need to populate this column for DELETED rows (we don’t need to populate it for the INSERTS or UPDATEs). When you DELETE a row in a DataTable you can’t make any further modifications to the row. Therefore we have to reject the change, which allows us to change the rows, and then update the column with a “D” for DELETED

 

DataTable ChangedDT = SourceDT.GetChanges (DataRowState.Added | DataRowState.Modified | DataRowState.Deleted|DataRowState);

DataColumn DelCol = ChangedDT.Columns.Add("action",typeof(string));

              

foreach (DataRow ChangedRow in ChangedDT.Rows )

{

    if (ChangedRow.RowState == DataRowState.Deleted)

    {

        ChangedRow.RejectChanges();

        ChangedRow ["action"] = "D";

    }

}

Now we have a DataTable only containing the changes and with our DELETED rows flagged with the D we need to create a SqlCommand object that will call the stored procedure we are using.

This is a normal SqlCommand with the commandText set to the name of the procedure. Creating the parameter is slightly different to normal, the SqlDbType of the parameter has to be SqlDbType.Structured. This is the type for all table valued parameters. In addition to this you have to specify the TypeName, this is used to say which TABLE TYPE your table valued parameter is. From my testing the TypeName is only required if your CommandType is NOT CommandType.StoreProcedure (So in the example below setting TypeName isn’t required), I guess this is because for StoredProcedures the type of the table is defined in the stored procedure definition.

 

Once the SqlCommand  is setup you can execute it as you would any other SqlCommand.

 

SqlCommand UpdateCommand = new SqlCommand();

UpdateCommand.Connection = conn;

UpdateCommand.CommandText = "dbo.up_OrderDetail_SaveSet";

   

UpdateCommand.CommandType = CommandType.StoredProcedure;

 

UpdateCommand.Parameters.AddWithValue("@ChangedOrderDetail", ChangedDT);

UpdateCommand.Parameters[0].SqlDbType = SqlDbType.Structured;

UpdateCommand.Parameters[0].TypeName = "SalesOrderDetail_Type";

 

conn.Open();

UpdateCommand.ExecuteNonQuery();

 

The stored procedure used by this code is derived from the MERGE example in the “Handling Deletes with MERGE” article.

 

The thing to note is that your TABLE TYPE has to have the additional column in it so the action can be passed through.

 

CREATE TYPE SalesOrderDetail_Type  as TABLE (

    [SalesOrderID]          [int] NOT NULL ,

    [SalesOrderDetailID]    [int] IDENTITY(1,1) NOT NULL,

    [CarrierTrackingNumber] [nvarchar](25) NULL,

    [OrderQty]              [smallint] NOT NULL,

    [ProductID]             [int] NOT NULL,

    [SpecialOfferID]        [int] NOT NULL,

    [UnitPrice]             [money] NOT NULL,

    [UnitPriceDiscount]     [money] NOT NULL,

    [LineTotal]             AS (isnull(([UnitPrice]*((1.0)-[UnitPriceDiscount]))*[OrderQty],(0.0))),

    [rowguid]               [uniqueidentifier] ROWGUIDCOL  NOT NULL,

    [ModifiedDate]          [datetime] NOT NULL,

    Action                  char(1),

    PRIMARY KEY (SalesOrderId,SalesOrderDetailID))

 

 

We then create our stored procedure to use this type,

 

 

CREATE PROCEDURE up_OrderDetail_SaveSet

  @ChangedOrderDetail SalesOrderDetail_Type READONLY

 ,@SalesOrderID       int

AS

 

    MERGE Sales.SalesOrderDetail OrderDetail

    USING @ChangedOrderDetail    Chng        ON Chng.[SalesOrderDetailID] = OrderDetail.[SalesOrderDetailID]

                                            AND Chng.[SalesOrderID]       = OrderDetail.[SalesOrderID]

    WHEN MATCHED AND Chng.Action = 'D' THEN

        DELETE

    WHEN MATCHED THEN

        UPDATE SET [SalesOrderID]            = Chng.SalesOrderID           

                  ,[CarrierTrackingNumber]   = Chng.CarrierTrackingNumber 

                  ,[OrderQty]                = Chng.OrderQty               

                  ,[ProductID]               = Chng.ProductID               

                  ,[SpecialOfferID]          = Chng.SpecialOfferID           

                  ,[UnitPrice]               = Chng.UnitPrice               

                  ,[UnitPriceDiscount]       = Chng.UnitPriceDiscount       

                  ,[rowguid]                 = Chng.rowguid               

                  ,[ModifiedDate]            = Chng.ModifiedDate           

    WHEN NOT MATCHED THEN

       INSERT ([SalesOrderID],[CarrierTrackingNumber],[OrderQty],[ProductID],[SpecialOfferID],[UnitPrice]

              ,[UnitPriceDiscount],[rowguid],[ModifiedDate])

       VALUES (Chng.SalesOrderID,Chng.CarrierTrackingNumber,Chng.OrderQty,Chng.ProductID

              ,Chng.SpecialOfferID ,Chng.UnitPrice,Chng.UnitPriceDiscount,Chng.rowguid,Chng.ModifiedDate);

 


This is cross posted from my SQLBlogcasts blog which can be found here, http://sqlblogcasts.com/blogs/simons/SQL Server 2008 - The MERGE and TVP data access pattern with DataAdapters

Categories:


Most applications are written nowadays using optimistic concurrency. Locks aren’t placed on a piece of data for the duration of someone viewing that data in their application. Instead what you do is when someone wants to save the data you check to see whether someone else as changed it since the time you read the data. If someone has you generally don’t want to save the data because you are likely to overwrite the change that had been made. The check normally consists of checking the value of each column against the values you got back when you read the data or comparing the value of a rowversion against what you read (a rowversion column gets changed every time a row is changed, that means you can use this column to detect a change rather than checking all the columns). Imagine buying a

What has this got to do with the MERGE statement. Well optimistic concurrency is very easy to do when you are dealing with a single row of data as you only have to do one check. If you identify the row has changed you can return success or failure. However with a MERGE statement what happens if you are updating 10 records and 4 have already been updated. In this case you have a number of options one is to update those that haven’t changed and return to the client those that have changed. This can be done very nicely using a trick with MERGE and the OUTPUT clause .

Assuming you have a rowversion (timestamp) column on your table (you can use the same approach if you don’t, in which case you will have to compare all the columns ) the following is an example MERGE statement with a TVP (Table Valued Parameter). We are going to save a set of OrderLines

The classic MERGE statement is as follows

MERGE Sales.SalesOrderDetail OrderDetail

USING @ChangedOrderDetail    Chng        ON Chng.[SalesOrderDetailID] = OrderDetail.[SalesOrderDetailID]

                                        AND Chng.[SalesOrderID]       = OrderDetail.[SalesOrderID]

WHEN MATCHED THEN

    UPDATE SET [SalesOrderID]            = Chng.SalesOrderID           

              ,[CarrierTrackingNumber]   = Chng.CarrierTrackingNumber 

              ,.....

WHEN NOT MATCHED THEN

    INSERT ([SalesOrderID],[CarrierTrackingNumber],[OrderQty],[ProductID],[SpecialOfferID]

           ,[UnitPrice],[UnitPriceDiscount],[rowguid],[ModifiedDate])

     VALUES

           (Chng.SalesOrderID,Chng.CarrierTrackingNumber,Chng.OrderQty,Chng.ProductID,Chng.SpecialOfferID

           ,Chng.UnitPrice,Chng.UnitPriceDiscount,Chng.rowguid,Chng.ModifiedDate);



To add optimistic checking you can check at the start within a serialised transaction or check at the end to see if the number of rows affected is the same as the number passed in i.e.

 

SET TRANSACTION ISOLATION SERIALIZABLE

BEGIN TRANSACTION

IF NOT EXISTS (SELECT 1

             FROM Sales.SalesOrderDetail OrderDetail

             JOIN @ChangedOrderDetail    Chng        ON Chng.[SalesOrderDetailID] = OrderDetail.[SalesOrderDetailID]

                                                    AND Chng.[SalesOrderID]       = OrderDetail.[SalesOrderID]

            WHERE OrderDetail.Rowversion <> Chng.Rowversion)

    BEGIN

        MERGE Sales.SalesOrderDetail OrderDetail

        USING @ChangedOrderDetail    Chng        ON Chng.[SalesOrderDetailID] = OrderDetail.[SalesOrderDetailID]

        .......

    END      

ELSE

    RAISERROR(‘data has changed since it was read....)

COMMIT TRANSACTION

Or

BEGIN TRANSACTION

 

    MERGE Sales.SalesOrderDetail OrderDetail

    USING @ChangedOrderDetail    Chng        ON Chng.[SalesOrderDetailID] = OrderDetail.[SalesOrderDetailID]

    .......

   

IF @@ROWCOUNT<> (SELECT Count(1) FROM @ChangedOrderDetail)

       BEGIN

    RAISERROR(‘data has changed since it was read....)

       ROLLBACK TRANSACTION

       END

ELSE

    COMMIT TRANSACTION

   

The final way is to do a partial update and report the records that have changed . The OUTPUT clause only contains records that have been inserted, updated or deleted. So to get the records out you need to UPDATE them, however you can UPDATE the columns to their existing values (i.e. not change anything ) and that way you can get these rows in the OUTPUT clause. You can then use a nice feature of the MERGE OUTPUT clause and compare the deleted rowversion value with that in the source data. If they are different it means the record wasn’t changed.

 

MERGE Sales.SalesOrderDetail OrderDetail

USING @ChangedOrderDetail    Chng        ON Chng.[SalesOrderDetailID] = OrderDetail.[SalesOrderDetailID]

                                      AND Chng.[SalesOrderID]       = OrderDetail.[SalesOrderID]

                                  

WHEN MATCHED THEN

    UPDATE SET [SalesOrderID]            = CASE WHEN Chng.Rowversion = OrderDetail.Rowversion          

                                                THEN Chng.SalesOrderID           

                                                ELSE OrderDetail.SalesOrderID

                                           END

              ,[CarrierTrackingNumber]   = CASE WHEN Chng.Rowversion = OrderDetail.Rowversion          

                                                THEN Chng.CarrierTrackingNumber 

                                                ELSE OrderDetail.CarrierTrackingNumber

                                           END

              , .....

 

OUTPUT CASE WHEN Chng.Rowversion = deleted.Rowversion THEN 1 ELSE 0 END Changed

      , Chng.SalesOrderDetailID

Note: For simplicity I have removed most of the columns from the update statement. You would need to include these in your application.

You can further extend this by using another new feature of SQL Server 2008, Composable DML. This allows you to consume the output from one statement into another statement. With SQL Server 2008 you are limited to only being able to consume the data in an INSERT statement. There are also restrictions on the structure of the table you are inserting into. Refer to BOL for exact details.

Using composable DML you can store only those records that failed update into a table variable and then return those records.

 

DECLARE @ChangedOrderDetail SalesOrderDetail_Type

DECLARE @SalesOrderID int

 

DECLARE @t TABLE (Changed int, ID int)

 

INSERT INTO @t

SELECT Changed, SalesOrderDetailID

FROM (MERGE Sales.SalesOrderDetail OrderDetail

      USING @ChangedOrderDetail    Chng        ON Chng.[SalesOrderDetailID] = OrderDetail.[SalesOrderDetailID]

                                     &n