Tuesday, July 2, 2013

SQL SERVER – Significance of Table Input Parameter to Stored Procedure


SQL Server has introduced a functionality to pass a table data form into stored procedures and functions. This feature greatly simplifies the process of developing. The reason being,  we need not worry about forming and parsing XML data. With the help of the table Input parameter to Stored Procedure we can save many round trips. Any SQL training will vouch for the fact that SQL is capable of accepting large, complex data in the form of parameters in a stored procedure.
Situation
Consider two tables
1) Sales Table
2)  SalesDetails Table
dbo.Sales
dbo.SalesDetails
In the Sales Table various products are there for a specific Sales Id. The SalesDetails Table also displays costs of these products along with them.
SaleID=1, for instance, has PurchaseOrderNumber=’BigOrder’. It has three products- Product1,Product2 and Product3.
We should note that SalesId in Sales Table functions as the primary key and function of secondary key is performed by SalesId in SalesDetails Table.
Whenever there is a new sale, we want to impart a unique and distinctive number to the sale in the sales table. Also, we should provide the SalesDetails Table with the sold products in the product column.
Explanation
We have to generate two stored procedures to provide a solution to this situation. We use the first stored procedure for inserting the data in the dbo.Sales table. With the second stored procedure we store the data in  dbo.SalesDetails table.
In case, we consider the situation displayed in the table above, we should use the first stored procedure only once for adding the data in the dbo.Sales table. To add the data in the dbo.SalesDetails table, we have to use the second stored procedure thrice. A total of four round trips occurs between the SQL server and application. The number of round trips can be reduced to one, if we use table Input parameter to Stored Procedure.
Step 1 – Open a new query window to the tempdb database
USE tempdb;
GO
Step 2 – Create a Sales and SalesDetails table
CREATE TABLE dbo.Sales
( SaleID INT IDENTITY PRIMARY KEY,
CustomerID INT,
PurchaseOrderNumber VARCHAR(20)
);
CREATE TABLE dbo.SalesDetails( SalesDetailID INT IDENTITY,SaleID INT REFERENCES dbo.Sales(SaleID),Description VARCHAR(50),Price DECIMAL(18,2)
);
GO
Step 3 – Create traditional insert stored procedures for both tables
CREATE PROCEDURE dbo.SalesInsert
@CustomerID INT,
@PurchaseOrderNumber VARCHAR(20),
@SaleID INT OUTPUT
AS BEGIN
INSERT INTO dbo.Sales (CustomerID,PurchaseOrderNumber)
VALUES(@CustomerID,@PurchaseOrderNumber);
SELECT @SaleID = SCOPE_IDENTITY();
END;
GO
CREATE PROCEDURE dbo.SalesDetailInsert
@SaleID INT,
@Description VARCHAR(50),
@Price DECIMAL(18,2),
@SalesDetailID INT OUTPUT
AS BEGIN
INSERT INTO dbo.SalesDetails (SaleID,Description,Price)
VALUES(@SaleID,@Description,@Price);
SELECT @SalesDetailID = SCOPE_IDENTITY();
END;
GO
Step 4 – Show how we would have previously inserted an order
Here four round trips will occur in which we call dbo.SalesInsert stored procedure once to insert the data into dbo.Sales and dbo.SalesDetailInsert stored procedure thrice to insert all the products for a particular  sales id
DECLARE @SaleID INT;
DECLARE @SalesDetailID INT;
BEGIN TRAN;
EXEC dbo.SalesInsert 12,'BigOrder',@SaleID OUTPUT;
EXEC dbo.SalesDetailInsert @SaleID,'Product 1',12.3,@SalesDetailID OUTPUT
EXEC dbo.SalesDetailInsert @SaleID,'Product 2',14.6,@SalesDetailID OUTPUT
EXEC dbo.SalesDetailInsert @SaleID,'Product 3',122.35,@SalesDetailID OUTPUT
COMMIT;
GO
SELECT * FROM dbo.Sales;
SELECT * FROM dbo.SalesDetails;
GO
Now we create a table data type
Step 5 – Create a table data type to hold the sales details
CREATE TYPE dbo.SalesDetails AS TABLE
( Description VARCHAR(50),
Price DECIMAL(18,2)
);
GO
Step 6 – Modify the insert procedure to take detail lines as well
ALTER PROCEDURE dbo.SalesInsert
@CustomerID INT,
@PurchaseOrderNumber VARCHAR(20),
@SalesDetails dbo.SalesDetails READONLY,
@SaleID INT OUTPUT
AS BEGIN
BEGIN TRAN;
INSERT INTO dbo.Sales (CustomerID,PurchaseOrderNumber)
VALUES(@CustomerID,@PurchaseOrderNumber);
SELECT @SaleID = SCOPE_IDENTITY();
INSERT INTO dbo.SalesDetails (SaleID,Description,Price)
SELECT @SaleID, Description,Price
FROM @SalesDetails;
COMMIT;
END;
GO
Step 7 – Perform an insert with a single round-trip
With the help of table data type to a stored procedure only one round trip is needed
DECLARE @SaleID INT;
DECLARE @SalesDetails dbo.SalesDetails;
INSERT INTO @SalesDetails VALUES('Product 1',12.3),('Product 2',14.66),('Product 3',122.35);
EXEC dbo.SalesInsert 12,'BigOrder',@SalesDetails,@SaleID OUTPUT;
GO
SELECT * FROM dbo.Sales;
SELECT * FROM dbo.SalesDetails;
GO
Table input parameter in SQL is a massive march ahead where development and potential performance are concerned. It can lessen server round trips, utilize table constraints and widen the functionality of programming on the database engine.



           

passing datatable to stored procedure

In this post, I will show you, how DataTable from C# is passed to a stored procedure in Sql Server. 
1.        Create a TableType with the structure as same of the DataTable in Sql Server 
2.        Make this TableType as input parameter of stored procedure. 
3.        Finally, pass the DataTable to the TableType parameter of the stored procedure

Note : This is supported on Sql Server 2008 onwards.

Step 1: First Create a Table Employee

CREATE TABLE Employee(
    EmpID        VARCHAR(10)
    , EmpName    VARCHAR(50)
    , Gender     CHAR(1)
    , DOJ        DATETIME 
)

Step 2 : Create a Table Type

CREATE TYPE EmpTableType AS TABLE (
    EmpID        VARCHAR(10)
    , EmpName    VARCHAR(50)
    , Gender     CHAR(1)
    , DOJ        DATETIME 
)

Step 3 : Create Stored Procedure that will take Table Type that we created in Step 2 as parameter 

CREATE PROCEDURE usp_GetEmpDetils(
    @EmpDet EmpTableType READONLY 
) AS 
BEGIN    
    INSERT INTO Employee
    SELECT * FROM @EmpDet
    
    SELECT * FROM Employee
END

Step 4 : ADO.Net Code passing DataTable to the TableType Parameter in Stored Procedure

DataTable EmpTable = new DataTable();
EmpTable.Columns.Add("EmpID");
EmpTable.Columns.Add("EmpName");
EmpTable.Columns.Add("Gender");
EmpTable.Columns.Add("DOJ");
DataRow EmpRow = EmpTable.NewRow();
EmpRow["EmpID"] = "EMP0001";
EmpRow["EmpName"] = "Sandeep Mittal";
EmpRow["Gender"] = "M";
EmpRow["DOJ"] = "01/01/2010";        
EmpTable.Rows.Add(EmpRow);
EmpTable.AcceptChanges();
 
SqlConnection connection = new SqlConnection("data source=ServerName;database=DBName;uid=UserID;pwd=Password");
SqlCommand selectCommand = new SqlCommand("usp_GetEmpDetils", connection);
selectCommand.CommandType = CommandType.StoredProcedure;
SqlParameter tvpParam = selectCommand.Parameters.AddWithValue("@EmpDet", EmpTable);
tvpParam.SqlDbType = SqlDbType.Structured;
connection.Open();
grid.DataSource = selectCommand.ExecuteReader();
grid.DataBind();
connection.Close();

Monday, July 1, 2013

SQL SERVER – 2008 – Introduction to Snapshot Database – Restore From Snapshot



SQL SERVER – 2008 – Introduction to Snapshot Database – Restore From Snapshot

Snapshot database is one of the most interesting concepts that I have used at some places recently.
Here is a quick definition of the subject from Book On Line:
A Database Snapshot is a read-only, static view of a database (the source database). Multiple snapshots can exist on a source database and can always reside on the same server instance as the database. Each database snapshot is consistent, in terms of transactions, with the source database as of the moment of the snapshot’s creation. A snapshot persists until it is explicitly dropped by the database owner.
If you do not know how Snapshot database work, here is a quick note on the subject. However, please refer to the official description on Book-on-Line for accuracy. Snapshot database is a read-only database created from an original database called the “source database”. This database operates at page level. When Snapshot database is created, it is produced on sparse files; in fact, it does not occupy any space (or occupies very little space) in the Operating System. When any data page is modified in the source database, that data page is copied to Snapshot database, making the sparse file size increases. When an unmodified data page is read in the Snapshot database, it actually reads the pages of the original database. In other words, the changes that happen in the source database are reflected in the Snapshot database.
Let us see a simple example of Snapshot. In the following exercise, we will do a few operations. Please note that this script is for demo purposes only- there are a few considerations of CPU, DISK I/O and memory, which will be discussed in the future posts.
  • Create Snapshot
  • Delete Data from Original DB
  • Restore Data from Snapshot
First, let us create the first Snapshot database and observe the sparse file details.
USE master
GO
-- Create Regular Database
CREATE DATABASE RegularDB
GO
USE RegularDB
GO
-- Populate Regular Database with Sample Table
CREATE TABLE FirstTable (ID INT, Value VARCHAR(10))
INSERT INTO FirstTable VALUES(1, 'First');
INSERT INTO FirstTable VALUES(2, 'Second');
INSERT INTO FirstTable VALUES(3, 'Third');
GO
-- Create Snapshot Database
CREATE DATABASE SnapshotDB ON
(Name ='RegularDB',
FileName='c:\SSDB.ss1')
AS SNAPSHOT OF RegularDB;
GO
-- Select from Regular and Snapshot Database
SELECT * FROM RegularDB.dbo.FirstTable;
SELECT * FROM SnapshotDB.dbo.FirstTable;
GO
Now let us see the resultset for the same.



Now let us do delete something from the Original DB and check the same details we checked before.
-- Delete from Regular Database
DELETE FROM RegularDB.dbo.FirstTable;
GO
-- Select from Regular and Snapshot Database
SELECT * FROM RegularDB.dbo.FirstTable;
SELECT * FROM SnapshotDB.dbo.FirstTable;
GO

When we check the details of sparse file created by Snapshot database, we will find some interesting details. The details of Regular DB remain the same.

It clearly shows that when we delete data from Regular/Source DB, it copies the data pages to Snapshot database. This is the reason why the size of the snapshot DB is increased.
Now let us take this small exercise to  the next level and restore our deleted data from Snapshot DB to Original Source DB.
-- Restore Data from Snapshot Database
USE master
GO
RESTORE DATABASE RegularDB
FROM DATABASE_SNAPSHOT = 'SnapshotDB';
GO
-- Select from Regular and Snapshot Database
SELECT * FROM RegularDB.dbo.FirstTable;
SELECT * FROM SnapshotDB.dbo.FirstTable;
GO
-- Clean up
DROP DATABASE [SnapshotDB];
DROP DATABASE [RegularDB];
GO
Now let us check the details of the select statement and we can see that we are successful able to restore the database from Snapshot Database.

We can clearly see that this is a very useful feature in case you would encounter a good business that needs it.
I would like to request the readers to suggest more details if they are using this feature in their business. Also, let me know if you think it can be potentially used to achieve any tasks.
Complete Script of the afore- mentioned operation for easy reference is as follows:
USE master
GO
-- Create Regular Database
CREATE DATABASE RegularDB
GO
USE RegularDB
GO
-- Populate Regular Database with Sample Table
CREATE TABLE FirstTable (ID INT, Value VARCHAR(10))
INSERT INTO FirstTable VALUES(1, 'First');
INSERT INTO FirstTable VALUES(2, 'Second');
INSERT INTO FirstTable VALUES(3, 'Third');
GO
-- Create Snapshot Database
CREATE DATABASE SnapshotDB ON
(Name ='RegularDB',
FileName='c:\SSDB.ss1')
AS SNAPSHOT OF RegularDB;
GO
-- Select from Regular and Snapshot Database
SELECT * FROM RegularDB.dbo.FirstTable;
SELECT * FROM SnapshotDB.dbo.FirstTable;
GO
-- Delete from Regular Database
DELETE FROM RegularDB.dbo.FirstTable;
GO
-- Select from Regular and Snapshot Database
SELECT * FROM RegularDB.dbo.FirstTable;
SELECT * FROM SnapshotDB.dbo.FirstTable;
GO
-- Restore Data from Snapshot Database
USE master
GO
RESTORE DATABASE RegularDB
FROM DATABASE_SNAPSHOT = 'SnapshotDB';
GO
-- Select from Regular and Snapshot Database
SELECT * FROM RegularDB.dbo.FirstTable;
SELECT * FROM SnapshotDB.dbo.FirstTable;
GO
-- Clean up
DROP DATABASE [SnapshotDB];
DROP DATABASE [RegularDB];
GO



Here are some notes on “SQL Server 2008 Database Snapshots” I took while attending an advanced class on SQL Server taught by Greg Low (from http://sqlblog.com/blogs/greg_low/ and http://www.sqldownunder.com/).
Please note that, although these notes were taken during the class, I might have added some of my own (mis)interpretation :-). Always check your facts on Books Online (I try to provide links when applicable). As with anything else you get from a blog, never use any of this in production before you thoroughly validate it a test environment. Please keep in mind that some of those will be hard to follow without some pre-requisite knowledge and the right context. Reading the post from top to bottom will help.

Database Snapshots
  • Read-only, static views of an entire database
  • See http://msdn.microsoft.com/en-us/library/ms175158.aspx
  • Used for: Reporting, protection from user error, major updates, unit testing
  • Careful – When using with major updates or bulk operations, consider the impact
  • Uses NTFS sparse files, always in the same server as database
  • CREATE DATABASE … ON (FILE=Name, FILENAME='...') AS SNAPSHOT OF …
  • See http://msdn.microsoft.com/en-us/library/ms175876.aspx
  • File specified under FILENAME contains changes in the original database since snapshot time
  • Can’t drop, detach or restore database (or add more files to it) when a snapshot exists
  • Careful - If you lose the original database files, you lose the snapshot 
Considerations 
  • Can create multiple snapshots of the same database
  • Careful – Additional write workload when updating original database with multiple snapshots
  • Can’t use with master, model or tempdb
  • Can’t change permissions on snapshot after it is created
  • No option to refresh a snapshot. Need to drop and recreate. 
  • If creating on mirrored database, the mirror needs to be synchronized
  • Created always with ALLOW_SNAPSHOT_ISOLATION set to ON
Demo
  • Create a database, create a table, insert a few rows
  • Create snapshot with CREATE DATABASE … AS SNAPSHOT OF …
  • Query the table in the snapshot
  • In Windows Explorer, at "Size" and "Size on Disk" properties of the file
  • Go back to original database, insert a few more rows, update some rows, delete some rows, query the table
  • Query the table in the snapshot - verify it has the old state
  • In Windows Explorer, at "Size" and "Size on Disk" properties of the file again

SSIS - delete files from a Network or Local path based on date

http://www.mssqltips.com/sqlservertip/2930/sql-server-integration-services-package-to-delete-files-from-a-network-or-local-path-based-on-date/

Problem

We have a requirement to delete a group of files that are older than the specified number of days from the company file share. This file share stores sensitive clients extracts (produced by DBAs as part of client’s SQL Server Agent extract jobs), database backups, scanned emails, etc. Moreover, due to the complex folder hierarchy and delicate nature of the data stored in these files, this task has to be originated from SQL Server. However, due to company security policy, and based on SQL Server security best practices, we blocked access to OLE Automation stored procedures, CLR features, and xp_cmdshell.  Is there any way to accomplish this task without using these features?  Check out this tip to learn more.

Solution

Well, this requirement can be fulfilled quiet easily by using the following SSIS toolbox components: Script Task, Foreach Loop container, and the File System Task. The following are the steps, which you can follow to accomplish this task using SSIS.

Steps to Build the SSIS Package for File Deletion

  • Launch the SQL Server Data Tools, then from the File menu, choose New, and click Project. This will launch the New Project dialog box.
  • In the New Project dialog box, select Integration Services Project from the templates pane. In the Name box, change the default name to DeleteRedundantFiles. Also, specify the project location, and once properly configured, click OK to create this new SSIS project.
  • By default, SQL Server Integration Service Project creates an empty package, named Package.dtsx, which is added to your project.  In the Solution Explorer, right-click on this package and rename it to DeleteRedundantFilesFromNetwork.dtsx.
using the following SSIS toolbox components

1: Defining Package Variables in SSIS

Next, add the following five variables to your SSIS package:
Variable Name: varFileAgeLimit
Variable Data Type: Int32
Description: This variable stores the age limit variable for the file. This variable is used within the Script Task, and accepts both positive and negative values. When a negative value is assigned to this variable, the code within the Script Task searches for files that are older than a specified number of days.  When the value is positive, the code within the Script Task searches the files that are newer than specified number of days.
Variable Name: varFileFQN
Variable Data Type: String
Description: This variable stores the file name with the path. For example: C:\DataFolder\Text.txt or \\MyServer\MyDataFolder\Test.txt.
Variable Name: varFilePattern
Variable Data Type: String
Description: The purpose of this variable is to store file name pattern. For example: *.* or *.bak or DB1_Extract.dat
Variable Name: varNetworkPath
Variable Data Type: String
Description: Stores either the Network location or the Local path, which is the code within the Script Task that will be used to search files in that location and within child directories of this location. For example: C:\MyLocation or \\MyServer\MyLocation.
Variable Name: varFileList
Variable Data Type: Object
Description: Stores the list of files that will be deleted.

2: Defining Package Tasks

Next, add and configure the following SSIS package tasks:

2.1: Configuring “Get File List – ST” Script Task

The first task you need for this solution is the Script Task. To add it to your package, simply drag it from SSIS Toolbox to design surface of the Control Flow tab. Now, on the Control Flow design surface, right-click on the newly added Script Task, then click Rename, and change the name from default name to Get File List – ST.
Next, double-click the Script Task, to open the Script Task Editor, and then configure the following properties:
  • Set the ScriptLanguage property to Microsoft Visual Studio C# 2010.
  • Add the varFileAgeLimit, varFilePattern and varNetworkPath package variables in the Script task’s ReadOnlyVariables property.
  • Add the varFileList package variable in the Script task’s ReadWriteVariables property.
  • Next, click the Edit Script button on the Script Task Editor page. This will open the VSTA integrated development environment (IDE) window. Copy the code below because this will search and return the list of files that are older than a specified number of days in a specified location, and within the child directories of this specified location. Once done, click the OK button on the Script Task Editor to save changes to the Script Task.
 double-click Script Task, to open Script Task Editor


Microsoft Visual Studio C# 2010 Code to Delete Files in the File System

#region Help:  Introduction to the script task
/* The Script Task search files in the specified location and within child directories of the specified location */
#endregion
#region Namespaces
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.IO; // Import for Directory class
using System.Collections; // Import for ArrayList class
#endregion
namespace ST_e10d8186ebb34debbc31bb734b0e29a5
{
[Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
public partial class ScriptMain : 
 Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
   private string NETWORK_PATH;
   private string FILE_PATTREN;
   private bool isCheckForNewer = true;
   int fileAgeLimit;
   private ArrayList listForEnumerator = new ArrayList();
   private void GetFilesInFolder(string folderPath)
   {
      string[] AllFiles;
      DateTime fileChangeDate;
      TimeSpan fileAge;
      int fileAgeInDays;
      try
      {
         AllFiles = Directory.GetFiles(folderPath, FILE_PATTREN);
         foreach (string fileName in AllFiles)
            {
               fileChangeDate = File.GetLastWriteTime(fileName);
               fileAge = DateTime.Now.Subtract(fileChangeDate);
               fileAgeInDays = fileAge.Days;
               CheckAgeOfFile(fileName, fileAgeInDays); 
            }
         if (Directory.GetDirectories(folderPath).Length > 0)
            {
              foreach (string childFolder in Directory.GetDirectories(folderPath))
               {
                  GetFilesInFolder(childFolder);
               }
            }
      }
      catch (Exception e)
      {
        System.Windows.Forms.MessageBox.Show("Exception caught: " + e.ToString(), "Results",
            MessageBoxButtons.OK, MessageBoxIcon.Error);
      }
   }
   private void CheckAgeOfFile(string fileName, int fileAgeInDays)
   {
     if (isCheckForNewer)
      {
        if (fileAgeInDays <= fileAgeLimit)
          {
             listForEnumerator.Add(fileName);
          }
      }
      else
      {
        if (fileAgeInDays > fileAgeLimit)
          {
              listForEnumerator.Add(fileName);
          }
      }
   }
   public void Main()
   {
     // Initializing class variables with package variables
     fileAgeLimit = (int)(Dts.Variables["User::varFileAgeLimit"].Value);
     NETWORK_PATH = (string)(Dts.Variables["User::varNetworkPath"].Value);
     FILE_PATTREN = (string)(Dts.Variables["User::varFilePattern"].Value); ;
     if (fileAgeLimit < 0)
     {
       isCheckForNewer = false;
     }
     fileAgeLimit = Math.Abs(fileAgeLimit);
     GetFilesInFolder(NETWORK_PATH);
     // Return the list of files to the variable
     // for later use by the Foreach from Variable enumerator.
     Dts.Variables["User::varFileList"].Value = listForEnumerator;
     Dts.TaskResult = (int)ScriptResults.Success;
   }
   #region ScriptResults declaration
   enum ScriptResults
    {
      Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
      Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
    };
    #endregion
  }
}

2.2: Configuring “Get Individual File FQN - FLC” Foreach Loop container

The next task you need is the Foreach Loop container. To add it to your package, drag it from the SSIS Toolbox to design the surface of the Control Flow tab. Now, on the Control Flow design surface, right-click the Foreach Loop container, then click Rename, and change the name from default name to Get Individual File FQN - FLC.
Next, double-click the Foreach Loop container, to open the Foreach Loop Editor. On left side of Foreach Loop Editor, click on the Collection, and then configure its properties:
  • Change Enumerator property to Foreach From Variable Enumerator.
  • Specify varFileList package variable as Enumerator configuration variable
Configuring “Get Individual File FQN - FLC” Foreach Loop container
Now, click on Variable Mappings and select varFileFQN package variable to map to the collection value.
click on Variable Mappings and select varFileFQN package variable
Once done, connect the Script task (Get File List – ST) with the Foreach Loop container (Get Individual File FQN – FLC).

2.3: Configuring “Delete Files on Remote Directory – FST” File System Task

Finally, add the File System Task to the Foreach Loop container (Get Individual File FQN – FLC). To do that, simply drag it from the SSIS Toolbox to the design surface of the Loop container (Get Individual File FQN – FLC). Then, right-click on the newly added File System Task, then click Rename, and change the name from default name to Delete Files on Remote Directory – FST.
Next, double-click on the File System Task, to open File System Task Editor, and then configure the following properties on General page:
  • Set Operation property to Delete file.
  • Set IsSourcePathVariable property to True.
  • Specify varFileFQN package variable as SourceVariable.
double-click File System Task, to open File System Task Editor
All done, our package is successfully configured, and it should look similar to figure below:
 our package is successfully configured

Testing

To test the package, simply assign values to the package variables, and then execute the package. For example, I specified the following values to package variables, to delete all files from \\JW02410\Temp\ shared folder that are older than 2 days.
assign values to package variables, and then execute the package


I specified the following values to package variables, to delete all files from \\JW02410\Temp\ shared folder that are older than 2 days
When I executed the package, it deleted all files from this location and within child directory of this location that are older than 2 days.
it deleted all files from this location