Visual Studio 2013 Database Project: Item Templates with VSIX

Visual Studio 2013 Database Project: Database Item Templates with VSIX

Overview

Strictly speaking VSIX is not a database project or related in kind, but by virtue of Item Templates.  With VSIX projects and the TemplateBuilder (NuGet) package, templates can be built from the contents of an ItemsTemplates folder.  The latter (if not created by TemplateBuilder) should be manually created as ItemTemplates folder as an item of the VSIX project.

NOTE: SQL Server (database) templates are not supported in Visual Studio 2013 VSIX and requires a manual effort to install on a developer’s IDE.  As indicated by the error item below, if the database files added to a VSIX project are changed to have a vstemplate file extension the compiler (on build) should issue this error saying that only CSharp and VBasic templates. Similarly if a Reference is added in the Asset tab of the manifest, the error The targer “TemplateProjectOuputGroup” does not exist in the project” is presented.

VS Database Template Support

VS Database Template Support

VSIX Template

The VSIX project template is available from the Visual Studio 2013 SDK and found under the menu: [Visual C#] >> Extensibility >> VSIX Project.

Template Builder

The TemplateBuilder package is installed from the Package Manager console or from Visual Studio: Tools >> NuGet Package Manager >> Package Manger Console.  Once installed, template are built to the folder: <Drive:>:\Users\<User>\Documents\Visual Studio 2013\Templates\ItemTemplates and <Drive:>\Users\<User>\Documents\Visual Studio 2013\Templates\ProjectTemplates.

NuGet Package Manager

Download and install the NuGet Package Manager for Visual Studio 2013.  It has a VSIX extension but installs by double-click and run.

SideWaffle

SideWaffle is a template pack that adds a selection of project and item templates to Visual Studio, including enabling templates to use for creating new ones. A VSIX project generates an experimental invocation of Visual Studio in which to test these templates so that the real instance is not affected.  To configure templates require a work-flow that is enabled by a separate application (that you create) to populate template and custom parameters when creating a new item.  This article, for now, mentions it but is not using it to create templates.

Create a New VSIX Project for Template Deployment

In order to author customised solutions, projects, code-snippets, and templates, a VSIX project is required.  Depending on the user-environment set-up for Visual Studio, file paths may vary slightly.

Create a new project:

  1. Follow the path: File >> New Project >> [Custom Path] >> [Visual C#] >> extensibility >> VSIX Project
  2. Provide a name for the project and check the solution details
  3. Fill out the VS VSIX Manifest with the appropriate details (reference)
  4. Install the TemplateBuilder package.
VS VSIX Manifest

VS VSIX Manifest

Installing TemplateBuilder

This installation requires the Package Manager Console (NuGet):

  • Follow the path: Tools >> NuGet Package Manager >> Package Manager Console.
  • Type Install-Package TemplateBuilder -Pre at the PM> prompt and hit [Enter].  NOTE: Internet access to https://www.nuget.org/api/v2/ is required.
VS Package Manager Console

VS Package Manager Console

The TemplateManager installation responds favourably on completion of the install, if it succeeds.  NOTE: The NuGet package is downloaded into the file location: <Drive>:\Users\<user name>\AppData\Local\NuGet\Cache and in the event of not having the default Internet location (https://www.nuget.org/api/v2/) available, Visual Studio looks here for whatever the package is being installed.

VS NuGet Installation of TemplateBuilder

VS NuGet Installation of TemplateBuilder

The installation process deposits an example template into: <Drive:>\Users\<User>\Documents\Visual Studio 2013\Templates\ProjectTemplates.  NOTE: there may not actually be an example template in this folder.

Create a Template Solution and Project

The Visual Studio Solution should contain the VSIX project as well as other projects, i.e. those used in which to create the templates.  For example, to create database templates a database project is added to the solution and the corresponding templates with parameters created in there.  This article uses the template created in a previous post (Visual Studio 2013 Database Project: Create a Stored Procedure Item Template) to illustrate how the VSIX project functions in this respect.

There are two methods:

  1. Exporting and importing a template (zip file)
  2. Referencing the template source project from the VSIX project where they are in the same solution

The latter (point 2) does not work for database templates

Exporting the Template

The template (or stored procedure created in the templates project) is created by exporting the necessary file (the actual stored procedure and any referenced files in the case of a project or c# type template) to the folder: <Drive:>\Users\<User>\Documents\Visual Studio 2013\My Exported Templates.  Follow the next steps to affect this:

  • Create, for example a stored procedure template (reference the above link on how to do that) an File >> Export Template >> Item Template, select the correct project, i.e. the database project containing the stored procedure created.
  • Select the correct stored procedure and complete the wizard.  The output template is locatable in the folder: <Drive:>\Users\<User>\Documents\Visual Studio 2013\My Exported Templates in a file called: <Template Name>.zip.

NOTE: Edit the MyTemplate.vstemplate file (contained inside the zip file) in Visual Studio and add an element TemplateGroupID (depicted in the code below) between the templateData tags to help with the filtering of template in the New Item dialog (reference).

VS Template Projects and Item Template

VS Template Projects and Item Template

1
 <TemplateGroupID>Database</TemplateGroupID>

The Item Template (Stored Procedure) is added to the VSIX project.  Use the .vsixmanifest file and add an Itemtemplate item to the Assets tab. Add a:

  • New item,
  • Pick ItemTemplate and
  • Point the File Location to the folder where the copied template zip file has been placed.

NOTE: The path could be: :\Users\\Documents\Visual Studio 2013\Templates\ItemTemplates\\Database\Programmability, but it is up to choice.  However, Visual Studio, on starting up, reads from this location on a user-instance, i.e. local machine.  The project is appended with a new folder, i.e. ItemTemplates and the zip file is appended.

On the  the property of the newly added template (zip) file, set the property Copy to Output Directory is set tot Copy Always.

NOTE: To inspect the XML code of the project file: right-click >> Remove and then right-click >> edit) reveals this setting in its XML structure, as depicted below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
    "ItemTemplates\Samples\StoredProcedure.zip">
      Always
      true
    
    "packages.config" />
    "Properties\template-builder.props" />
    "source.extension.vsixmanifest">
      Designer
    
  

NOTE: Turn off the Build and Deploy options in the Configuration Manager for the VSIX project.

Using a Reference to the Template Project in the same Solution

A reference to the template project (the one where the stored procedure is authored) is added to the Reference of the VSIX project by using the source.extension.vsixmanifest file’s Assets tab, as before.  But, in this case, the reference is to a project in the same solution.  The fields [Type],

[Source], and [Project] are completed, with project being the name of the said template project.  This adds a reference to the VSIX project.

Set the Properties of the reference (see how), especially the VSIX Sub Path that determines where the template is installed on the menu-tree of the New Item menu function.

NOTE: Visual Studio does not provide a project template for creating database objects, so it seems the export method is the only way in which to get the zip file into the VSIX for deployment.  This aspect may be resolvable by creating the zipped files in a database project and let the project zip them like the wizard and place in the appropriate folder.  Not yet sure how to do this.

Build the Project and Install the Templates

The VSIX project is built and it copies the template zip file to the output folder: \Users\\Documents\Visual Studio 2013\Projects\\\bin\Debug as a .vsix (installable) file along with a TemplateBuilder.dll and the folder structure (Visual Studio) that contains the actual template file.

NOTE: For developer projects, this installation file should be made available on a network shared drive and installed as updates are made, or from a Private Gallery reference) that notifies users via RSS feeds of new extensions.  Alternatively the same file can be uploaded to the Visual Studio Gallery for community consumption and addition.

file in the VSIX project to set the [Assets] by creating a new reference to an ItemTemplate of a project (the one containing the stored procedure template) that also adds it to the References of the VSIX project. Here the Properties of the added assembly can be altered to achieve a variety of things. See the screen image (below) for a visual explanation of what is described.

VS VSIX Template Database Item Folder Location Reference

VS VSIX Template Database Item Folder Location Reference

Executing the <Name>.vsix (installable) file invokes all the things that were set in the manifest to install the template(s).  These are available in Visual Studio and can be inspected via: Tools >> Extensions & Updates >> Templates
, illustrated below.

VS Extensions and Updates Templates folder

VS Extensions and Updates Templates folder

Using the newly installed Templates

After installing the VSIX template package and verifying the extension is in Visual Studio, the template can be used from which to create Projects or Items.  During the installation the templates are deployed to a random folder: %LocalAppData%\Microsoft\VisualStudio\12.0\Extensions and by virtue of %LocalAppData% it only applies to the current user.  Alternatively, setting the manifest to AllUsers=true the extension is installed under Common7\IDE\Extensions (where VS is installed) and appears to all users on the computer (reference).

Visual Studio 2013 Database Project: Working the Project LocalDbs

Visual Studio 2013 Database Project: Working the Project LocalDbs

Overview

Visual Studio (VS) has evolved much to what it is capable of today for database development and deployment.  With the addition of a much better table create facility Visual Studio is the development tool of choice for the database developer.  SQL Server Management Studio still has its place as a light free tool, but for serious database development with Team Foundation Integration, Visual Studio is the tool of choice.

Using Local Databases

Visual Studio creates local databases when a database project is created, viz.

  1. MSSQLLocalDb — SQL 2014 local db
  2. ProjectsV12 — created by SQL Server Data Tools (SSDT)
  3. V11.0 — SQL 2013 local db
VS SQL Server Object Explorer

VS SQL Server Object Explorer

These local database instances are located on a file location on the local drive of the computer that runs Visual Studio at: C:\Users\hendriekusr\AppData\Local\Microsoft\Microsoft SQL Server Local DB\Instances.

The server name consequently is: (localdb)\ProjectV12.  Details about these installed databases can be obtained by running the SQlLocalBD.exe (located in C:\Program Files\Microsoft SQL Server\110\Tools\Binn\) against the name of the database, e.g. info ProjectsV12.

VS Local Database Information

VS Local Database Information

Use the Instance pipe name (from above execution) to connect to a local database from SQL Server Management Studio (SSMS), as depicted below.

SSMS Local Db Connection Dialog

SSMS Local Db Connection Dialog

A connection (to view database objects) is possible from VS and SSMS with the latter using the pipe connection string.  The SSMS screen (below) illustrates how the connection is made to the localDB as well as a Stored Procedure created from a SSMS template that was added to the database.  The next article [Create a Stored Procedure Item Template] illustrates how this same SSMS Stored Procedure template is created for Visual Studio.

SSMS Connection to LocalDB with Stored Procedure from Template

SSMS Connection to LocalDB with Stored Procedure from Template

Previous Article (2) [Create Database Project Folders]. . . Next Article (4) [Create a Stored Procedure Item Template]

Visual Studio 2013 Database Project: Create a Stored Procedure Item Template

Visual Studio 2013 Database Project-Create a Stored Procedure Item Template

Overview

Using Visual Studio for database development requires that database objects like stored procedures be created, amongst other items such as tables, view, triggers, etc.  It is advantageous if creating an item that the proposed script is a template that already conforms to the company’s standards, by providing a boiler-plate script that speeds up the developer’s options.  Even better would be if these templates are available to all developers as they check items in-and-out of Team Foundation Server.  Bear in mind that a particular organisation might have a single development office or location, but others may be geographically dispersed.  It would be expedient if all of them are singing from the same hymnal.

This article explores the basics of Visual Studio Templates for database development.  Using this phrase as a search term, an Internet search engine divvies up a few articles in support.  One of which is called An Overview of Templates for Database and Server Objects (Microsoft).  It suggests that existing templates can be customised to affect all templates used by the team.  Customisable project item/ stubs accelerate development by providing the developer with a head-start. Templates are comprised of project and item templates.  Every time a user opens a menu item in Visual Studio to create something, a template is invoked, e.g. New Project, or Add New Item.

Templates exist in the user community (How to Add Items Using Online Item Templates) or a VSIX project with SideWaffle (Code4Fun), but these aren’t discussed here.  To Illustrate the concept, a template for a Stored Procedure is created and added as an Item Template to the local developer machine.  This is absorbed into Visual Studio and provides the developer with a new item in the Add New Item option.

Creating the Template

A Stored Procedure (via Add New Item ) in Visual Studio (VS) and for that matter in Microsoft SQL Server Management Studio (SSMS) is pretty uneventful.  Therefore, should a developer create quite a few items, the work required to customise these to the company standard absorbs unnecessary time and effort.  A template should make this much easier.

Either use VS or SSMS to create a stored procedure that conforms to all general company standards.  This one to be used in Visual Studio, so the usage between SSMS and VS is different from a parameterisation perspective.

In SSMS the code-block (below) illustrates how parameter are coded and then on invocation provided by Ctrl+Shift+M.  In Visual Studio this is done through an XML configuration and only what the respective dialog requests on New Item is user-provided, e.g. the project or object name.

1
2
3
4
5
6
CREATE PROCEDURE	[<Schema_Name, sysname, Schema_Name>].[<Procedure_Name, SYSNAME, ProcedureName>] 
					<@Param1, SYSNAME, @p1> <Datatype_For_Param1, , INTEGER> = <Default_Value_For_Param1, , 0>
				,	<@Param2, SYSNAME, @p2> <Datatype_For_Param2, , INTEGER> = <Default_Value_For_Param2, , 0>
				,	<@Param3, SYSNAME, @p3> <Datatype_For_Param2, , INTEGER OUTPUT>
WITH EXECUTE AS [CALLER	| SELF | 'user_name'] -- Default: CALLER				
AS

The flowing code-block represents the Visual Studio equivalent of the obove code for the same purpose.  The $fileinputname$ is the file name provided by the developer when creating a new stored procedure.  All the other parameters are configured in the Item Template’s XML file.  Therefore, the XML configuration should prescribe the purpose of the stored procedure, I.e. suit exactly what it is being created for. 

1
2
3
4
5
6
CREATE PROCEDURE	[$SchemaName$].[$fileinputname$] 
					$ProcParam1$	$ProcParam1DataType$	=	$ProcParam1Value$
				,	$ProcParam2$	$ProcParam2DataType$	=	$ProcParam2Value$
				,	$ProcParam3$	$ProcParam3DataType$		$ProcParam3Type$
WITH EXECUTE AS [CALLER	| SELF | 'user_name'] -- Default: CALLER <-- make a selection in the executing principle context.				
AS

A Visual Studio Item Template for a Custom Stored Procedure

The provided template fast-tracks a developer and provides a consistent standard for every stored procedure created base on this template.  This particular standard is comprised of a few sections, viz.

  • Create with execution principal context
  • Flower box containing the abstract and owner details
  • Initial proc-wide directives, e.g. NOCOUNT, EXACABORT, and TRANSACTION ISOLATION
  • Error-handling variables (for RAISERROR)
  • Local variable declaration & initialisation
  • Main query execution block wrapped in TRY. . . CATCH and transaction
  • CATCH block with detailed error handling with XACT_STATE
  • Return value for application event handling
  • Directive switch to system defaults
  • Metadata section for object details (metadata) when it’s built in the database
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
CREATE PROCEDURE	[$SchemaName$].[$fileinputname$] 
					$ProcParam1$	$ProcParam1DataType$	=	$ProcParam1Value$
				,	$ProcParam2$	$ProcParam2DataType$	=	$ProcParam2Value$
				,	$ProcParam3$	$ProcParam3DataType$		$ProcParam3Type$
WITH EXECUTE AS [CALLER	| SELF | 'user_name'] -- Default: CALLER <-- make a selection in the executing principle context.				
AS
/* ==================================================================================================================================
	ABSTRACT:	This procedure executes for $projectname$ and. . . 
	VERSION:	$ItemVersion$ (MAJOR.MINOR.PATCH)

	OWNER:		$ItemOwner$
	TEMPLATE:	$TemplateVersion$ (MAJOR.MINOR.PATCH)

	<NOTE: to developer, complete the metadata at the end of this procedure script.>

											<< See bottom of item for details >>
=================================================================================================================================== */
BEGIN
	SET NOCOUNT ON;
	--SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
	--				READ COMMITTED
	--				REPEATABLE READ
	--				SNAPSHOT
	--				SERIALIZABLE;
	
	-- --------------------------------------------------------------------------------------------------
	--	Renders entire txn uncommittable if constraint violation occurs. 
	--	ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10de_6tsql/html/cbcaa433-58f2-4dc3-a077-27273bef65b5.htm
	-- --------------------------------------------------------------------------------------------------
	SET XACT_ABORT ON;
	
	-- ------------------------------------------------------------------------------------
	--	VALIDATE INCOMING PARAMETERS
	-- ------------------------------------------------------------------------------------
	-- None.
		
	-- -------------------------------------------------------------------------------------
	--	DECLARE LOCAL VARIABLES
	-- -------------------------------------------------------------------------------------
	-- Error Handling Local Variables
	-- Delete this for SQL 2014 unless required for custom error logging
	DECLARE			@vErrorNumber		INTEGER	=	0
				,	@vErrorMessage		NVARCHAR(2048) = N''
				,	@vErrorSeverity		INTEGER = 0
				,	@vErrorState		TINYINT = 0
				,	@vErrorLine			INTEGER = 0
				,	@vErrorProcedure	NVARCHAR(256) = NULL;
	
	-- Processing Local Variables
	DECLARE			@LocalVariable1Name	INTEGER
				, 	@LocalVariable2Name	DATETIME;
			
	-- -------------------------------------------------------------------------------------
	--	INITIALISE LOCAL VARIABLES
	-- -------------------------------------------------------------------------------------
	
	-- Initialise processing variables
	SET			@LocalVariable1Name	=	0;
	SET			@LocalVariable2Name	=	'2011-01-01';

	/* =========================================================================================
		MAIN CODE EXECUTION
	============================================================================================ */
	BEGIN TRY
		BEGIN TRANSACTION;
		-- { sql_statement | statement_block }
		COMMIT TRANSACTION;
		RETURN(@vErrorNumber);
	END TRY
	BEGIN CATCH
		/* ------------------------------------------------------------------------------------------
			ERROR HANDLING
			Severity 0-10 (information) are not caught by TRY. . . CATCH
			Severity 11-16 (user-correctable)
			Severity 17 (Insufficient Resources)
			Severity 18 (non-fatal error)
			Severity 19 (SQL resource error)
			Severity 20-25 (fatal system error) execution termination with SQL close
			https://msdn.microsoft.com/en-us/library/ms164086.aspx

			Any sys.messages valid messages can be utilised but must be passed to RAISERROR or THROW.
		-------------------------------------------------------------------------------------------- */

		-- Grab error indicators.
		-- Delete this for SQL 2014 unless required for custom error logging
		SELECT			@vErrorNumber		=	ERROR_NUMBER() 
					,	@vErrorLine			=	ERROR_LINE()
					,	@vErrorSeverity		=	ERROR_SEVERITY()
					,	@vErrorState		=	ERROR_STATE()
					,	@vErrorProcedure	=	ERROR_PROCEDURE()
					,	@vErrorMessage		=	ERROR_MESSAGE()+ CHAR(10)
												+ N'Executing code item name or type ('+ISNULL(@vErrorProcedure,N'AdHoc SQL')
												+ N'), error number ('+ CONVERT(NVARCHAR(20), @vErrorNumber)
												+ N'), severity ('+CONVERT(NVARCHAR(20), @vErrorSeverity)
												+ N'), code error line ('+CONVERT(NVARCHAR(20),@vErrorLine)										
												+ N').'+CHAR(10) +CHAR(13);
		-- -------------------------------------------------------------------------
		--	END TRANSACTION CORRECTLY.		
		--	XACT_STATE =  1 ~ Transaction is committable.
		--	XACT_STATE = -1 ~ Transaction is uncommittable - roll back 
		--	XACT_STATE = 0 ~ No transaction - commit or rollback  results in error.
		--	ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10de_6tsql/html/248df62a-7334-4bca-8262-235a28f4b07f.htm
		--	https://msdn.microsoft.com/en-us/library/ms175976.aspx
		--  Any custom error loggin should be implemented here
		-- -------------------------------------------------------------------------
		IF (XACT_STATE() = 1) 
		BEGIN
			COMMIT TRANSACTION;
			PRINT N'Transaction('+ISNULL(ERROR_PROCEDURE(),N'AdHoc SQL')+N'): COMMITTED';
			RETURN(@vErrorNumber);
		END ELSE
		BEGIN
			IF (XACT_STATE() != 0) ROLLBACK TRANSACTION;
			SET @vErrorMessage	= @vErrorMessage + N'Transaction: ROLLED BACK';

			-- SQL 2000+
			RAISERROR	(	@vErrorMessage		-- { msg_id | msg_str | @local_variable }
						,	@vErrorSeverity
						,	@vErrorState		-- [ ,argument [ ,...n ] ] )  [ WITH option [ ,...n ] ]
						)
			-- SQL Server 2014+
			--THROW; 
			RETURN(@vErrorNumber);
		END
	END CATCH;

	-- Cleanup/ Housekeeping
	SET TRANSACTION ISOLATION LEVEL READ COMMITTED; -- Database Default
	SET XACT_ABORT OFF;
	SET NOCOUNT OFF;
END

/* ==================================================================================================================================
	TEMPLATE AUTHOR: $TemplateAuthor$, COMPANY: $OwnerCompany$

	OBJECT VERSION HISTORY
	AUTHOR:	<Name.Last Name>, ORGANISATION: <Company Name>, AUTHORED DATE <YYYY-MM-DD>

	VERSION		DATE		NAME SURNAME		CHANGE DETAILS
	---------	-----------	-----------------	---------------------------------------------------------------------------------------
	$ItemVersion$		YYYY-MM-DD	Name.Surname		Change details are.....

												<< See bottom of item for details >>
=================================================================================================================================== */
GO
/* ---------------------------------------------------------------------------------------------
	OBJECT METADATA
	NOTE:	This section has to be edited and run for all attributes after each edit to keep the
			database metadata up to date.
	 Add extended properties to the view (in database objects)
		1. Author name
		2. View naming specifics
		3. View version
		4. Editor name
		5. Edit date
		6. Template version
 ----------------------------------------------------------------------------------------------- */

-- Author details
EXECUTE sys.sp_addextendedproperty	@name = N'Author Name'
				,	@value = N'$ItemAuthor$' --<-- Change this only!
				,	@level0type = N'SCHEMA'
				,	@level0name = N'$SchemaName$'
				,	@level1type = N'PROCEDURE'
				,	@level1name = N'$fileinputname$'	
				,	@level2type = NULL
				,	@level2name = NULL;
GO				

-- Object description
EXECUTE sys.sp_addextendedproperty	@name = N'Functional Description'
				,	@value = N'This procedure executes for $projectname$ and. . .' --<-- Change this only!
				,	@level0type = N'SCHEMA'
				,	@level0name = N'$SchemaName$'
				,	@level1type = N'PROCEDURE'
				,	@level1name = N'$fileinputname$'	
				,	@level2type = NULL
				,	@level2name = NULL;
GO

-- This view version
EXECUTE sys.sp_addextendedproperty	@name = N'Version'
				,	@value = N'$ItemVersion$' --<-- Change this only!
				,	@level0type = N'SCHEMA'
				,	@level0name = N'$SchemaName$'
				,	@level1type = N'PROCEDURE'
				,	@level1name = N'$fileinputname$'	
				,	@level2type = NULL
				,	@level2name = NULL;
GO

-- Editor details
EXECUTE sys.sp_addextendedproperty		@name = N'Editor Name'
				,	@value = N'$EditorName$' --<-- Change this only
				,	@level0type = N'SCHEMA'
				,	@level0name = N'$SchemaName$'
				,	@level1type = N'PROCEDURE'
				,	@level1name = N'$fileinputname$'
				,	@level2type = NULL
				,	@level2name = NULL;
GO					
-- Edit date
EXECUTE sys.sp_addextendedproperty		@name = N'Edit Date'
				,	@value = N'$EditDate$' --<-- Change this only!
				,	@level0type = N'SCHEMA'
				,	@level0name = N'$SchemaName$'
				,	@level1type = N'PROCEDURE'
				,	@level1name = N'$fileinputname$'
				,	@level2type = NULL
				,	@level2name = NULL;	
GO
-- Template version
EXECUTE sys.sp_addextendedproperty	@name = N'Template version'
				,	@value = N'$TemplateVersion$' --<-- Change this only!
				,	@level0type = N'SCHEMA'
				,	@level0name = N'$SchemaName$'
				,	@level1type = N'PROCEDURE'
				,	@level1name = N'$fileinputname$'
				,	@level2type = NULL
				,	@level2name = NULL;
GO
-- End --------------------------------------------------------------------------------------

Configuring the Stored Procedure Template in Visual Studio

The template can be created in any SQL editor, viz. SSMS or VS.  Once it is created, open Visual Studio and add a Stored Procedure to an existing Project, using the default VS functionality.

VS Add new Item to Project

VS Add new Item to Project

Follow these steps:

  1. Select Stored Procedure and create it by giving it a name that would suit the template Stored Procedure.
  2. Take the newly created template SQL code for Stored Procedure (listed in the code-block above) and paste it over the text of the newly created item.
  3. Save it and then open: File >> Export Template
  4. Select Item Template radio-button from the [Export Template Wizard] and the appropriate project into which the template Stored Procedure has been saved
  5. Click the [Next] button
  6. Select the name of the Stored Procedure just created and saved in the project
  7. There should be no Item References to select so click the [Next] button
  8. Provide a Template Name, description, Icon, Preview Image, and deselect the Automatically import the template into Visual Studio (reference), keep the Display an explorer window on the output files folder selected
  9. Click the [Finish] button, and the output files folder is presented

A zip-file is created that contains three files:

  1. TemplateIcon.ic
  2. MyTemplate.vstemplate
  3. <Stored Procedure>.sql (the template SQL)

The MyTemplate.vstempate file is the one used to configure how the stored procedure is created, i.e. provide the name, description, input parameters, and other functions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
<VSTemplate Version="3.0.0" xmlns="http://schemas.microsoft.com/developer/vstemplate/2005" Type="Item">
  <TemplateData>
    <DefaultName>ACreateStoredProcedure.sql</DefaultName>
    <Name>ACreateStoredProcedure</Name>
    <Description>Default CreateStoredProcedure for Acme Corporation</Description>
    <ProjectType>SQLDB</ProjectType>
    <SortOrder>10</SortOrder>
    <Icon>__TemplateIcon.ico</Icon>
  </TemplateData>
  <TemplateContent>
    <References />
    <ProjectItem ReplaceParameters="true" TargetFileName="$fileinputname$.sql" >Z Create Stored Procedure.sql</ProjectItem>
    <CustomParameters>
	<CustomParameter Name="$projectname$" Value="API"/>
        <CustomParameter Name="$SchemaName$" Value="dbo"/>
	<CustomParameter Name="$ProcParam1$" Value="@InputParameter1"/>
	<CustomParameter Name="$ProcParam1DataType$" Value="INTEGER"/>
	<CustomParameter Name="$ProcParam1Value$" Value="0"/>
	<CustomParameter Name="$ProcParam2$" Value="@InputParameter2"/>
	<CustomParameter Name="$ProcParam2DataType$" Value="INTEGER"/>
	<CustomParameter Name="$ProcParam2Value$" Value="0"/>
	<CustomParameter Name="$ProcParam3$" Value="@OutputParameter"/>
	<CustomParameter Name="$ProcParam3DataType$" Value="INTEGER"/>
	<CustomParameter Name="$ProcParam3Type$" Value="OUTPUT"/>
	<CustomParameter Name="$ItemVersion$" Value="0.0.1"/>
	<CustomParameter Name="$TemplateVersion$" Value="0.0.1"/>
	<CustomParameter Name="$ItemOwner$" Value="c 2015 Acme (Pty) Ltd."/>
	<CustomParameter Name="$OwnerCompany$" Value="Acme (Pty) Ltd."/>
	<CustomParameter Name="$TemplateAuthor$" Value="Hendriekus J.Roux"/>
	<CustomParameter Name="$ItemAuthor$" Value="Hendriekus J.Roux"/>
	<CustomParameter Name="$EditorName$" Value="Hendriekus J.Roux"/>
	<CustomParameter Name="$EditDate$" Value="2015-03-09"/>
    </CustomParameters>
  </TemplateContent>
</VSTemplate>

The listing of the MyTemplate.vstemplate (above) illustrates two parameter functions, viz. system parameter and custom parameters. These are placed in the template SQL code as place-holders that are filled-out either by providing a name in response to an action in Visual Studio like New >> Project or New >> Item where a name is provided by the developer.  Or, the custom parameters are filled-out in correspondence to the values provided in the MyTemplate.vstemplate file.  The latter should be considered in affinity to the functional intension of the Item, I.e. Stored Procedure in this case.

 Present the Template Available in Visual Studio

Visual Studio present a dialog when a new Stored Procedure is created.

VS Create New Item Dialog Option List

VS Create New Item Dialog Option List

The template should be presented here in a section appropriate to the company and project.  To create a new section the template is added to a folder location on the developer’s machine.  The Visual Studio custom templates are located here: <drive>:\Users\<user name>\Documents\Visual Studio 2013\Templates\ItemTemplates.

VS Custom Templates Folder

VS Custom Templates Folder

By adding templates either into these locations or by creating a new folder, say Acme Items, any new custom templates could be placed there and become available from the New Item dialog.

In this example create a folder here called: ItemTemplates\Acme Corporation\Programmability into which the zip-file (<drive>:\Users\<user>\Documents\Visual Studio 2013\My Exported Templates) is copied.

To use the new template, restart Visual Studio and add a new Stored Procedure to the template using the new template.

VS Custom Template New Item Dialog

VS Custom Template New Item Dialog

Unfortunately, at this point, none of the parameters (except the procedure name) is substituted in the actual SQL code.  These, i.e. the custom parametes must yet be added to the MyTemplate.vstemplate file.

Adding Custom Parameters to a Template

Follow these steps to add these parameters and their values:

  1. Extract the template’s zip file from its location into the [My Exported Templates] location and open the  file in Notepad
  2. Take the parameters between the <CustomParameters> tag from the code (above) and add it to the current template’s file in the appropriate location
  3. Save the file and drag the saved version from the explorer window over the zip file in the template location  and choose the overwrite option to up date the template
  4. Restart Visual Studio and create a new Stored Procedure from the template to see all parameters being populated.

Previous Article (3) [Create a Database Project with Folder Structure]. . . Next Article (4) [Coming Soon]

Visual Studio 2013 and Database Projects: Create a Database Project with Folder Structure

Visual Studio 2013 and Database Projects: Create a Database Project with Folder Structure

Overview

Data base projects in Visual Studio is a bit of an after thought on application development.  It started with Gert Drapers (the data dude) and became SSDT (SQL Server Data Tools) that had to be installed (as an add-on) to Visual Studio.  This had some varying function over the years, but was aimed at reducing the difficulty of database development in Visual Studio and keep database changes in synch across a solution (matching database and application versions) and multiple server environments through the application life-cycle.

Creating a Database Project

This is quite easy using Visual Studio 2013.  On the first landing page (can be turned off) a Start section has the option from which to create a new project, or the traditional FILE menu is just as simple.

Visual Studio for Database Development First Screen

Visual Studio for Database Development First Screen

VS Create New Database Project Dialog

VS Create New Database Project Dialog

Be mindful that if a project is created and Team Foundation Server (TFS) is being used to take care of how the solution and projects contained by the solution are demarcated.  It is not in the scope of this article to explain how to do this, but suffice to say that where solutions are stored separately from projects, to first create an empty solution and check it in.  Then the projects that are added back to the solution so that all are checked into TFS in their correct locations.

Further take not of the Visual Studio boxes, viz. Solution Explorer, SQL Server Object Explorer, and Properties for example.  These should be orientated (docked) in positions that make sense, I.e. for easy reference.  Below is an example of an instance that contains a Solution with three projects.

VS Database Project Screen

VS Database Project Screen

The two most important boxes are the Solution Explorer and the SQL Server Object Explorer.  The former depicts the Solutions and the database projects.  Where no database actually exists (this is where you create it) SQL Server Objects can be created/ added to the project, or (where a database already exists) imported by a few means.  The first is using a DACPAC (SQL Server Data-tier application extract file), or connecting directly to a SQL Server database, or create one using a script file containing all the object definitions.

Choosing a Folder Structure

Importing a database allows for the folder structure in the project to be changed.  The project folder structure should allow for ease of maintenance and readability.  The folder structure does not influence deployment.

VS Database Project Import Dialog Folder Structure

VS Database Project Import Dialog Folder Structure

An import of an existing database (where no structure is chosen, or defined dumps all files in the root of the project, which is moderately messy, so it must be carefully considered.  A folder is not created where an imported database does not have objects, e.g. Stored Procedures.  If subsequent objects are required, the folder will first have to be created.

Organising the database advances readability, consequently also maintenance, because developers are able to find things quicker.  Any database objects without any discernable grouping are dumped in the root of the project on import.

NOTE:Things that impact the display of objects and what access rights are differentiated in practice, bears on the configuration of the naming scheme.  Making use of database schemas (to group objects and assign login permissions) is an effective strategy for data architecture over using prefixes on objects like sp or pr for a Stored Procedure (source).  The folder structure in Visual Studio and SQL Server Management Studio already informs the object type and with an added schema denotes an organisational name space for such objects.  SQL Server uses a four-part naming scheme: Server.Database.DatabaseSchema.DatabaseObject.  Objects in a schema are owned by the security principle that owns the schema and a schema can contain objects owned by different users.  Any predefined schema (dbo, sys, etc) can be dropped from the model database if not required consequently preventing them from being created in new databases.  A schema owner can be changed without breaking the code, but if an object owner, say Tommy.Object is removed, an ownership change must first be performed before Tommy’s account can be deleted.  However, any fully qualified references will break in the code, e.g.  Server.Database.[tommy[.DatabaseObject. The default dbo schema is owned by the dbo user. Users assigned to the dbo schema do not inherit dbo user account permissions, but the database objects contained in the schema (source).  Using dbo as the owner of all object can greatly simplify managing objects (source).

Previous article (1) [Installation]. . . Next Article (3) [Working the Project]

Visual Studio 2013 and Database Projects: Installation

Visual Studio 2013 and Database Projects–Installation

This installation is for Visual Studio Premium.  Once the image is mounted, the folder contains the installation file for the install, i.e. vs_premium.

VS Premium Folder

VS Premium Folder

Execute the vs_premium file and agree to the License Terms and Privacy Policy, and the location of the installation. Two options are quite important:

  1. Light Switch
  2. Microsoft SQL Server Data Tools

Light Switch is a technology to shorten the development time on business applications and data services for data-centric business applications.  SQL Server Data Tools are the add-on previously (VS 2012) downloaded and installed separately, but now part of the install.  It provides database schema modelling and support for querying live databases.

The other default option is Blend, a visual user-interface design tool using XAML (web) and WYSIWYG (desktop), before part of the Microsoft Expression Studio toolset.  This is not a database specific tool.

Once installed (it takes a while) the Microsoft Account Login dialog is presented for which you are required to have credentials.  Either use an existing account or create one (reference) and then associate an MSDN subscription (licenses that your company pays for) to this identity. Note: a licence key may also be used if you don’t have an MSDN subscription to activate the software (see how).

Microsoft Account Login

Microsoft Account Login

Once logged in access your account details to find the MSDN section where the Subscriber ID and company email are used to link the MSDN profile to the Microsoft User Identity.

Once the subscription is activated the software is licensed.

MSDN Subscription Activation Dialog

MSDN Subscription Activation Dialog

Visual Studio starts with an environment set-up that can be changed at a later stage also.  Here the Database environment (as that is the purpose of this instance) is selected and the colour scheme of your choice.

VS Environment Set-up Dialog

VS Environment Set-up Dialog

This is the installation process and you have arrived at the first screen.

Visual Studio for Database Development First Screen

Visual Studio for Database Development First Screen

Next article in series. . . 2 [coming soon]

Navigate. . . Next Article (2)

Microsoft® SQL Server™ AlwaysOn™ for Performance Mitigation?

Executive Summary

Microsoft® SQL Server™ AlwaysOn™ is a database clustering technology intended for high availability (HA) and Disaster Recovery (DR) with the ability to accommodate other beneficial workloads, such as read-only reporting query load sharing and database backups.

This article investigates using AlwaysOn™ as a performance optimisation (load sharing) mechanism for read-only reporting purposes.  In conclusion and by association with the intended purpose of AlwaysOn™, this function is not a preferred performance mitigation strategy for database access in an enterprise.  However, this does not imply that performance enhancements are not achievable, quite the contrary.

Firstly, AlwaysOn™ is not intrinsically designed for performance mitigation (between OLTP and Reporting workloads) reasons and the mitigation of concurrency, although it does provide secondary benefits to alleviate such contentions.  SQL Server 2012 can be positioned as a less-performant database system, when compared to SQL Server 2014, simply because of technological advancements in the latter.  This, specifically because of SQL Server 2014’s in-memory (Hekaton) ability and the ColumnStore indices of the Data Warehouse, amongst many new enhancements.

However, Hekaton is an OLTP beneficial enhancement, maybe even resulting in the removal of the need to have a secondary database over which to defray concurrency by sharing the load, as a result of the significant performance gains achieved by the technology.  However, AlwaysOn™ for geographically dispersed databases, is a superb solution for brokering regionalised access to data, discharging connections and load across the database system.  But, for a single enterprise architecture, mitigating performance for reporting style read-only queries, either by federation or a replication architecture, assuming that AlwaysOn™ as the perfect replacement, is erroneous.

Great performance gains are achievable through indexing secondary databases, via the primary, but requires them to be the same on both, despite different workloads.  Also, despite the additional performance benefits of the temporary query statistics in the tempdb and the compounded performance gains of SQL Server 2014, it does not herald AlwaysOn™ as the de facto load balancing mechanism for offloading reporting workloads by virtue of the database’s ability.  For this reason, it is not a preferable mechanism to spawn active secondary databases in lieu of any existing replicated databases, just because we can, thinking this is how to upgrade our replicated model.  It is probably more possible that by using SQL Server 2014 with Hekaton, increasing the hardware specification that the replicated databases can be abolished altogether, because of a much increased native ability of the SQL Server 2014.  With its in-memory abilities, any[1] amount of transactional or reporting queries should be tolerated by the database server.  Rationalising a proper enterprise data access strategy, so as to avoid the unnecessary implementation of business logic (modelling) on the database, is achievable through other technologies in augmentation of SQL Server 2014’s new performance abilities, e.g. Data Virtualisation, which is only as good as the underlying database performance.  AlwaysOn™ should not be seen as the panacea to these problems, because of the technology enhancements of the actual database server.

This article concludes that AlwaysOn™ should be used for its intended purpose, i.e. High Availability and Disaster Recovery, and in a limited capacity to defray concurrency and performance, except in the case of geographically dispersed database.  It should not be applied to mitigate performance issues between OLTP and Reporting queries as the preferred method, but rather that a more apt enterprise data access strategy be architected to benefit enterprise database performance issues.


[1] Factors more than with SQL Server 2012, so much so that we could almost (by comparison) say any amount of queries.

Introduction

SQL Server® is a Microsoft™ database technology for enterprise relational database management systems.  It declares a feature called AlwaysOn™ that is essentially a High Availability, for planned and unplanned disaster recovery purposes, something akin to (other high availability options) such as replication, log shipping, sql clustering and database mirroring.  AlwaysOn™ addresses the functional needs that had been acquired from the aforementioned architectural patterns, in an attempt to augment and simplify their respective limitations, by providing the following abilities:

  • Database Replicas, up for five, one primary and four[1] secondary stored
  • High Availability and Disaster Recovery, as an enhanced mirroring with read-only access
  • Synchronous and asynchronous transactions between primary (R/W) and replicas (R), with replicas using snapshots to avoid blocking
  • Use replicas for reporting access
  • Automatic failover

A combination of the best of failover clustering and database mirroring is the new HA/ DR feature of an Availability Group[2] of the AlwaysOn™ function.  An Availability Group is a group of databases that fails over, together (as a unit) from one replica of SQL Server to another replica within the same availability group (Ali, 2012).  Each availability replica resides on a different node of a single Windows Server failover cluster (WSFC)

This article considers only the viability of using AlwaysOn™ for the purpose of offsetting reporting workloads[3].

To realise how to leverage AlwaysOn™ for performance optimisation is to understand how the primary and replica databases can be used for this purpose.

Active Secondary Capabilities

Using active secondary[4] capabilities provides better resource utilisation (lowers cost) by offloading read intent applications[5] and backup jobs to secondary replicas.  This reduces the load on a primary[6] database (AlwaysOn Availability Groups (SQL Server), 2012).  The readable secondary, although providing read-only access, is not set to read-only, but is dynamic.  On a typical secondary, the data is near real-time, i.e. only a few seconds (Active Secondaries: Readable Secondary Replicas (AlwaysOn Availability Groups), 2012).

The deflection of workloads from the primary database is achieved by directing read-only connections to a readable secondary replica, but such a connection should tolerate latency.  To remove blocking, the read-only workloads use row versioning[7] and are automatically mapped to a snapshot isolation transaction level, in spite of the actual explicit setting, and in ignorance of locking hints.  Read-only routing is achieved by employing an availability group, listener to which the client directs the connection, as a read-only intent. There is no such thing as load balancing[8] because the connection is routed to the first available and readable secondary on the routing list of the primary.   A secondary database can influence performance on the primary because of the snapshot isolation transaction level modus. Ghost record lean-up on the primary can be blocked by transactions on a secondary.  For example (in an extreme case) if a secondary database has a long running query or when it gets disconnected or the data movement is suspended, it prevents log truncation, the secondary may have to be removed from the availability group to release the primary database’s holding pattern[9]. (Active Secondaries: Readable Secondary Replicas (AlwaysOn Availability Groups), 2012).

Indexing & Statistics

To deflect workloads to a replica in the availability group, for reporting purposes, yields a query pattern that is significantly different from a transactional database.  Presenting a more optimal performance yield, requires that the replica be able to curtail its response in a pro-reporting query fashion.  Schema or data changes, on a secondary database, must come from the primary, because of the read-only attribute of the replica.  Creating the indices on the primary allows the redo process to transfer them to the replicas.

To optimise query plans on the read-only replicas, different from that on the primary (but in addition to those on the primary) the statistics on the secondary is created in tempdb[10].  These statistics are temporary, as read-only statistics that are only created by SQL Server, but can be dropped by the user.

SQL Server 2014 a Better Option?

Almost always, a standard line from a Microsoft consultant is, “upgrade to the next version” for every existing version’s weaknesses.  So, what does SQL Server 2014 offer for performance mitigation between OLTP and reporting workloads, such as can be achieved in SQL Server 2012 with AlwaysOn™?  SQL Server 2014’s in-memory database ability[11] (Hekaton) is integrated with AlwaysOn™ by having the tables also readable in the secondary[12] (Diaconu, et al., 2013).  Hekaton, however is an OLTP[13] optimisation, using in-memory direct pointers (in place of disk-stored-indices), not indirectly such as page/ row ids to address data.  Hekaton is integrated into SQL Server 2014. (Larson, Zwilling, & Farlee, 2013).

But, does Hekaton benefit an active secondary for the purpose of offloading query pressure from the primary?  There are a few major Hekaton improvements:

  • In-memory tables[14]
  • Indices, hash and range with direct pointers to data, all in memory[15]
  • Stored Procedures compiled to native machine code
  • High level of concurrency[16] not relying on portioning, using latch free data structures to avoid physical interference amongst threads with a new optimistic, multi-version concurrency control technique to reduce interference among transactions. (Larson, Zwilling, & Farlee, 2013).

From a published AlwaysOn™ features perspective, SQL Server 2014 provides the following enhancements over SQL Server 2012:

  • Enhanced Availability for read-only Replicas
  • Increased Number of Replicas
  • Integration with Windows Azure
  • Enhanced Diagnostics

These don’t, on face value, translate to any material benefit over SQL Server 2012, by just looking at what’s new in AlwaysOn™.  But, if the advances of Hekaton is considered, having all of those features available in the secondary databases, provides a significant performance ability yield, but that by virtue of the database engine and not because of AlwaysOn™.

The Column Store Index option[17] (Data Warehousing) is part of the in-memory ability of SQL Server 2014 providing greater data compression for more real-time analytics support.  The resource governor of SQL 2014 also provides the ability to perform IO management (Team, SQL Server 2014: A Closer Look, 2013).  It also includes buffer-pool-extension support for solid state drives, enabling faster paging (Foley, 2013).  All of these, yet again, are database engine optimisations and enhancements, and not AlwaysOn™.  So, any performance gains are because of the database technology and not as a result of the AlwaysOn™ feature and any metrics thus achieved, should be attributed to the database technology only.


[1] This is a SQL Server 2012 constraint that is doubled with SQL Server 2014 (Otey, 2013).

[2] Availability groups cannot be created across domains, nor against multiple clusters in the same domain, nor having two replicas on the same node, part of the same group, neither replicate system databases, neither perform cross database transactions in a group (Golla, 2012).

[3] Scaling read-only workloads to replicas is available only in the SQL Server Enterprise Edition and attracts a full new server licence for every active replica. “When the secondary server does any work at all, like taking requests or running backups off of it, or any of the readable features of AlwaysOn are enabled then it must be fully licensed” (Hughes, 2012).

[4] Change tracking and change data capture are not available on secondary databases that belong to a readable secondary replica.

[5] This is very advantageous specifically for geographically distributed instances of a databases

[6] In SQL Server 2014, the relationship between the primary and the replica is more stable, in that if the primary goes down, the replica is still usable (Ozar, 2013).

[7] Row versioning increases data storage in the primary and secondary, although version data is not generated by the primary database.  A read-committed snapshot isolation (RCSI) level setting is turned on for the primary database.

[8] In SQL Server 2014, load balancing can be implemented using simple DNS round-robin or specialised load balancing solutions, either hard or software based. (Team, AlwaysOn in SQL Server 2014 CTP1, 2013).

[9] However, if ghost records (on the primary) is still needed by the secondary, a DBCC SHRINKFILE on the primary database log may fail.

[10] Any server restart flushes all read-only replica statistics customised for the read queries.

[11] In-memory performance is achievable across OLTP, DW and BI (Underwood, 2013).

[12] Failover time is not dependent on size of durable memory optimised tables (Underwood, 2013).

[13] OLTP-like, i.e. short lived transactions, high degree of concurrency, e.g. stock trading, order processing, or travel reservations.  Long running transactions should be avoided.

[14] Max size SCHEMA_AND_DATE = 512 GB (Underwood, 2013)

[15] In the event of a failure, all tables and indices are rebuilt entirely from the latest checkpoint and logs.

[16] Concurrency is obtained through multi-versioning per update per user.

[17] Column Store Indices can be updated for real-time data without dropping and recreating them (Belnekar, 2013).  In-memory ColumnStores provide fast execution for data warehouse queries and eliminates the need for other indices.  Microsoft’s PDW makes use of the in-memory ColumnStore in its Delta Store, converting data into a columnar format once the segment (1M rows) is full and is the preferred storage engine of the PDW. (Underwood, 2013).

Works Cited

Active Secondaries: Readable Secondary Replicas (AlwaysOn Availability Groups). (2012). Retrieved from Microsoft Technet: http://technet.microsoft.com/en-us/library/ff878253.aspx

Ali, A. (2012, January 19). Using The AlwaysOn Feature of SQL Server 2012. Retrieved from SQL Server Performance: http://www.sql-server-performance.com/2013/alwayson-clustering-failover/

AlwaysOn Availability Groups (SQL Server). (2012). Retrieved from Microsoft Technet: http://technet.microsoft.com/en-us/library/hh510230.aspx

Belnekar, P. (2013, June 18). Priti Belnekar’s Blogs. Retrieved from Blogspot: http://pritibelnekar.blogspot.com/2013/06/sql-server-2014-new-features.html

Diaconu, C., Freedman, C., Ismert, E., Larson, P.-A., Mittal, P., Stonecipher, R., . . . Zwilling, M. (2013). Hekaton: SQL Server’s Memory-Optimized OLTP Engine. Redmond: Microsoft. Retrieved from http://research.microsoft.com/pubs/193594/Hekaton%20-%20Sigmod2013%20final.pdf

Foley, M. (2013, June 11). Microsoft’s SQL Server 2014: More than just in-memory OLTP. Retrieved from ZDNet: http://www.zdnet.com/microsofts-sql-server-2014-more-than-just-in-memory-oltp-7000016672/

Golla, K. (2012, September 17). SQL Server 2012: Always On FAQs. Retrieved from Blogs MSDN: http://blogs.msdn.com/b/srgolla/archive/2012/09/17/sql-server-2012-always-on-faqs.aspx

Hughes, J. (2012, October 17). Does Sql Server 2012 Always On support Active/Passive? Retrieved from Stack Exchange: http://dba.stackexchange.com/questions/27088/does-sql-server-2012-always-on-support-active-passive

Larson, P.-A., Zwilling, M., & Farlee, K. (2013). The Hekaton Memory-Optimized OLTP Engine. Redmond: Microsoft. Retrieved from http://sites.computer.org/debull/A13june/Hekaton1.pdf

Otey, M. (2013, September 20). Changes to SQL Server 2014 AlwaysOn Availability Groups. Retrieved from SQL Seerver Mag: http://sqlmag.com/sql-server-2014/changes-sql-server-2014-alwayson-availability-groups

Ozar, B. (2013, June 03). (Almost) Everything You Need to Know About SQL Server 2014. Retrieved from Brent Ozar: http://www.brentozar.com/archive/2013/06/almost-everything-you-need-to-know-about-the-next-version-of-sql-server/

Team, S. (2013, June 27). AlwaysOn in SQL Server 2014 CTP1. Retrieved from SQL Server Blog: http://blogs.technet.com/b/dataplatforminsider/archive/2013/06/27/alwayson-in-sql-server-2014-ctp1.aspx

Team, S. (2013, June 05). SQL Server 2014: A Closer Look. Retrieved from SQL Server Blog: http://blogs.technet.com/b/dataplatforminsider/archive/2013/06/05/sql-server-2014-a-closer-look.aspx

Underwood, J. (2013, September 22). SQL Server 2014 In-Memory, High Performance and Mission Critical Overview. Retrieved from Slideshare: http://fr.slideshare.net/idigdata/sql-server-2014-inmemory-and-mission-critical

SQL Server Data via Enterprise Library 5.0

Introduction

This is a very simple Enterprise Library 5.0 example.

Getting started with reading data from SQL Server, via Enterprise Library 5.0, requires that the framework be installed on the developing machine and added to the Project References, and  ‘using’ references, of the development environment, e.g. Visual Studio 2012.

The software is available from MSDN [1]: Enterprise Library 5.0 — May 2011 (Build 5.0.414.0), but it is recommended that you use NuGet [5] to put the Enterprise Library onto the solution in Visual Studio.  This way, the latest version of the Enterprise Library section is downloaded and installed, e.g. 5.0.505.0.  I am not sure why the MSDN link points to Enterprise Library (for a stand-alone Msi download) an older version(5.0.414.0) and NuGet has the latest, i.e. 5.0.505.0.

NOTE: Using NuGet to get the Enterprise Library in the: Tools >> Library Package Manager >> Manage NuGet Packages for Solution >> Online >> All, then type ‘entlib’ into the Search Online (Ctrl+E) box, and hit enter button.

Install the following packages from NuGet:

  • Microsoft.Practices.EnterpirseLibrary.Common
  • Microsoft.Practices.EnterpirseLibrary.Data
  • Microsoft.Practices.EnterpirseLibrary.ServiceLocation
  • Microsoft.Practices.EnterpirseLibrary.Unity
  • Microsoft.Practices.EnterpirseLibrary.Unity.Configuration
  • Microsoft.Practices.EnterpirseLibrary.Unity.Inception
  • Microsoft.Practices.EnterpirseLibrary.Unity.Inception.Configuration

In Visual Studio, using the menu path: Tools >> Library Package Manager >> Package Visualizer, you can get a listing of all the packages installed (see image below) for this solution.

Package Manager Visualiser

Package Manager Visualiser

Using Enterprise Library leverages functionality from a collection of reusable application blocks to assist developers with common development objects.   Amongst many, the common block are Data Access, Exception Handling, Logging, Security and Validation.  To get to the NuGet packages (online), as well as see which ones have been installed, in addition to the above method, use the Visual Studio menu path: Tools >> Manage NuGet Packages for Solution… The image (below) lists all packages (you can type entlib into the search Online field) with a green-tick indication of those you have already installed.

NuGet Package Manager for Solution

NuGet Package Manager for Solution

This article focuses on the Data Access Application Block.

Problem

Rewriting common code, for every specialised implementation of application code, is not very optimal because it is error prone, because it does not lend itself to reproducing exactly the same implementation.  This creates a maintenance dependency, as errors may vary in the different code blocks that essentially do the same thing.  A standardised layer of optimised code, substituting common boiler-plate code plumbing, should be used.  This layer is the Microsoft Enterprise Library.

This article looks at standardising database access through the enterprise library.

Solution

Enterprise Library Installation

Download and install Microsoft Enterprise Library, or use NuGet (recommended).  Create a Visual Studio 2012 console application project, to test a data retrieval.  Add references of the Enterprise Library to the project, or if you have used NuGet, these are added automatically to the solution.

NOTE: NuGet adds a new config file, i.e. packages.config, to track installed packages and versions, but, this file is not part of the project and should not go into source control.

Configuration & Code

Once a reference has been added to the project, have a look at the Codeplex [2] examples for download.  From this example: [EntLib Dev Guide RC – Code Samples.zip], look in the DataAccess > DataAcess folder for the App.config file.

Two sections are important, viz. configSections, and connectionString.  These have to be added, with the former being configured to point at the installed dataConfiguration, i.e. Enterprise Library.  The latter is the connection string to the actual database.  See the next section on how to use the Enterprise Library Config. tool, to create/ change the app.config file.  This is a whole lot less error prone than rolling your own.

Code (from the Microsoft example):

<configuration>
    <configSections>
        <section name="dataConfiguration" type="Microsoft.Practices.EnterpriseLibrary.Data.Configuration.DatabaseSettings, Microsoft.Practices.EnterpriseLibrary.Data, Version=5.0.505.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" requirePermission="true" />
    </configSections>
<dataConfiguration defaultDatabase="TradeInformation" />
<connectionStrings>
    <add name="TradeInformation" connectionString="Database=TradeInformation;Server=TradeDatabasseServer;Integrated Security=SSPI"
        providerName="System.Data.SqlClient" />
</connectionStrings>
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/></startup></configuration>

NOTE: Instead of editing the configuration file manually, it can also be done via a tool that provides a user-screen.  Simpy right-click on the configuration file in the project (the one referring to Enterprise Library) and select Edit Configuration File, from the menu list.  For more detail, see the MSDN example [3]: Using the Configuration Tools.  If this option does not show up (right-click) make sure you have the version that targets Visual Studio 2012, in stalled: http://visualstudiogallery.msdn.microsoft.com/029292f0-6e66-424f-8381-3454c8222f9a [4], restart Visual Studio.

EnterpriseLibrary.Config Right-Click

EnterpriseLibrary.Config Right-Click

When using the Enterprise Library.Config (right-click the app.config) application to configure the data access configuration.  The user-interface requires that a Block (Add Data Settings) be added.  In the screen-grab (below) the app.config file can be seen, with the EnterpriseLibrary.Config user-interface, used to create it with below.  Doing it this way eliminates human configuration errors.

Enterprise Library.config

Enterprise Library.config

The defaultDatabase setting is a default connection, chosen by the GetInstance() method, of the connection listed first, below, when no database key is specified for the method.   The second example illustrates a connection, made by specifying the database connection key/ name.

_db = EnterpriseLibraryContainer.Current.GetInstance();
_db = EnterpriseLibraryContainer.Current.GetInstance("TradeInformation");

NOTE: If you get the error “The OutputPath property is not set for this project” when building the solution, make sure that the Configuration Manager (Debug, Release), as well as the Properties of every project in the solution are targeting the same CPU, e.g. x64.  Also refer to [http://gabrielmagana.com/2010/04/solution-to-the-outputpath-property-is-not-set-for-this-project/] and [http://connect.microsoft.com/VisualStudio/feedback/details/518181/msbuild-gets-confused-with-build-configuration-and-starts-generating-output-path-errors], if this is not the problem.

Code Example

Using the Codeplex example, I changed it a bit to make a simple Enterprise Library database call to defaultDB over which an ExecuteReader loops to print each record (in this case: trade) out to the console window.

I had a few issues with the .dll file references of Enterprise Library files.  The short of it is that when these come into the project via NuGet, they are copied (the packages) into the  c:users<your profile>DocumentsVisual Studio 2012Projects<project name>packagesEnterpriseLibrary.Common.5.0.505.0libNET35<.dll name>.   A copy also resides in the bin folder:

  • Microsoft.Practices.EnterpriseLibrary.Common.dll
  • Microsoft.Practices.EnterpriseLibrary.Data.dll
  • Microsoft.Practices.Unity.Configuration.dll
  • Microsoft.Practices.Unity.dll
  • Microsoft.Practices.Unity.Interception.Configuration.dll
  • Microsoft.Practices.Unity.Interception.dll

The project references point to the “packages” folder (build), but the application references the bin folder, at run-time.  Make sure that you have these references pointing to the correct .dll files for the Enterprise Library.  You could find yourself with endless problems when the defaultDB = EnterpriseLibraryContainer.Current.GetInstance(); attempts to get the database instance, using the Enterprise Library. It will either complain about not being able to load the correct .dll files, or that the database key (referring to the database name of either your named or default database in the app.config file) could not be loaded.

Code (Program.cs):

Code:

using System;
using System.Data;
using System.Data.Common;
using System.Data.SqlClient;
using System.ComponentModel;
using Microsoft.Practices.EnterpriseLibrary.Common.Configuration;
using Microsoft.Practices.EnterpriseLibrary.Data;

namespace za.co.acme.DataAccess
{
    class Program
    {
        static Database defaultDB = null;
        static void Main(string[] args)
        {

            defaultDB = EnterpriseLibraryContainer.Current.GetInstance<Database>();
            ReadTradesToConsole();
        }
        private static void ReadTradesToConsole()
        {
            String tSQL = "SELECT TradeID, ReceivedSequenceNumber, TradeDateTime, TradePrice, TradeSize FROM [dbo].[TradeInformation]";
            using (IDataReader reader = defaultDB.ExecuteReader(CommandType.Text, tSQL))
            {
                while (reader.Read())
                {
                    for (int i = 0; i < reader.FieldCount; i++)
                    {
                        Console.WriteLine("{0} = {1}", reader.GetName(i), reader[i].ToString());
                    }
                    Console.ReadLine();
                }
            }
        }
    }
}

The Codeplex exmaple provides a list of different usage patterns for the data, e.g. reading data from a stored procedure (with named/ unamed parameters), reading data as objects/ XML/ Scalar value, even doing all of this asynchronously. The asynchronous methods check the Enterprise Library’s public virtual bool SupportsAsync { get; } method, to see if the operation is possible, before attempting it.

Citations

No. Author Web Site Name URL Date Authored Date Cited
 1.  Microsoft  MSDN  http://msdn.microsoft.com/en-us/library/ff632023.aspx  May 2011  28 January 2013
2.  Microsoft  CodePlex  http://entlib.codeplex.com/  14 December 2012  28 January 2013
3.  Microsoft  MSDN: Using the Configuration Tools  http://msdn.microsoft.com/en-us/library/ff664633(v=pandp.50).aspx  na  28 January 2013
4.  Microsoft  MSDN: Visual Studio Geleray – EnterpriseLibrary.Config http://visualstudiogallery.msdn.microsoft.com/029292f0-6e66-424f-8381-3454c8222f9a  na  14 March 2013
5.  NuGet Docs  NuGet Frequently Asked Questions  http://docs.nuget.org/docs/start-here/nuget-faq  na  15 March 2013

Last Updated: 19 March 2013 09h19

Visual Studio 2010/ 2012: Model and Query Database Objects

Introduction

Executing a Visual Studio Database Project’s queries against an existing database, is somewhat different between the two versions, viz. Visual Studio (VS.) 2010 and 2012.  It is slight but useful to consider, after updating both versions to the latest version of SQL Server Data Tools, to see what these are.  This article is not an exhaustive account, but endeavours to jump-start a database developer on the differences.

Prerequisites

Software Versions

Visual Studio

The version of Visual Studio 2012 is upgraded to Service Pack 1 [1, 2], and all references to functionality is relative to that version.

NOTE: Remember to stop any anti-virus software before running the service pack, otherwise the error: “Access is denied” is presented.  The file name of the update is: vsupdate_KB2707250.exe [2], with the updated (after installation) build number:  04940-004-0038003-02092 .  It takes just under forty five minutes to complete.

NOTE: Ensure that the latest SQL Server Data Tools (for Visual Studio 2012) is installed, after this update [5].  If not, an incompatibility message is displayed by Visual Studio on the SQL Server Object Explorer tab, where database connections are listed.  The update installation takes about ten minutes, with the build number updating to: 11.1.21208.0.  After about seven minutes the installation dialogue hangs for a few minutes, with all the bars green, don’t kill it, it will complete.  The same applies for Visual Studio 2010, with build number: 10.3.21208.0, to download from the same location.

Microsoft SQL Server

The version of the installed 2012 database, is upgraded to Service Pack 1, bringing it to build: 11.0.3000.00.  The name of the image file is: en_sql_server_2012_sp1_x86_x64_dvd_1176976.iso, which is obtainable from your MSDN subscription.  NOTE: Remember to turn any anti-virus software off, as explained above.  The installation stalls for a bit, giving no indication that it is actually still running, however, a process called scenarioengine.exe (Windows Task manager) is active, and associated with the currently running update.  A few minutes later (with scenarioengine.exe resting on a zero percent CPU utilisation), a window suddenly pops up.  The SQL Server 2012 Service Pack 1 pop-up, should execute the update for all installed features, after agreeing to the licensing stop.  It takes about an hour.

For SQL Server 2012, the table below provides a list of updates at the time of writing.  NOTE: See the citation [3] on how to determine the installed version, and get to the latest update(s) on the web.

Build Knowledge Base (KB) Description URL Date Released
 11.0.3000.00  KB2674319  Service Pack 1  http://support.microsoft.com/kb/2755533  09 November 2012
 11.0.2383.00  KB2758687  Cumulative Update 4  http://support.microsoft.com/kb/2758687  15 October 2012
 11.0.2332.00  KB2723749  Cumulative Update 3  http://support.microsoft.com/kb/2723749  31 August 2012
 11.0.2325.00  KB2703275  Cumulative Update 2  http://support.microsoft.com/kb/2703275  18 June 2012
 11.0.2316.00  KB2679368  Cumulative Update 1  http://support.microsoft.com/kb/2679368  12 April 2012
 11.0.2100.60  na  SQL Server 2012 RTM  na  06 March 2012

NOTE: Visual Studio can install a SQL Express database known as LocalDB, so that a separate instance of SQL Server does not have to be maintained by developers on their machines [4].

The Problem

Tool-bars, Commands, and going about functionality differ slightly between Visual Studio versions (for database developers), potentially making the navigation an unfamiliar experience.

The Solution

Provide examples of how a database can be accessed, using Visual Studio 2012, in comparison with the older 2010 version, of the development environment, to accelerate the learning experience.

Model and Query Database Objects in Visual Studio 2012

Because we have just moved from Visual Studio 2010 (or having never used Visual Studio, but SQL Server Management Studio) for database development, it is expedient to know the differences and consequential enhancements, when working with a database.

To model or query database objects, as in SQL Server Management Studio, using Visual Studio, is accomplished in several ways.  A designer or query can be used to create and/ or change database objects.

Visual studio can connect to a database, without having a database project open, to view and change objects on the database.  With a project opened, objects can be maintained (in the project tree) while being connected to the database.

The experienced difference, between just working on a database through Visual Studio, or working on a database through Visual Studio’s Database Project, lies in the connection state to the database.  The latter is implicit, by means of the project’s properties, and the former explicit, by connecting to a server and database for the purpose of executing queries that could perform various tasks.

[SQL] Server [Object] Explorer

This function can be used without having a database project open, and also on the side, when working with an open project.  A database structure-tree, similar to that in SQL Server Management Studio, presents itself in the navigation pane of this function.  Commands can be issued, from this pane, against objects in the database.

[SQL] Server [Object] Explorer is a tab that is accessed from the menu tool-bar, or by short-cut key (for both VS. versions–with square brackets denoting the VS. 2012 variance in the name), as follows:

  • View >> Server Explorer
  • Ctrl+Alt+S

From the [SQL] Server [Object] Explorer tab, a new connection is added:

Visual Studio Server Explorer Comparison

Visual Studio Server Explorer Comparison

Visual Studio 2010
  • Use the menu path (from the right-click menu on Data Connections):  Add Connection, or
  • Use the Visual Studio 2010 Add Connection icon
Visual Studio 2012
  • User the menu path  (from the right-click menu on SQL Server): Add SQL Server, or
  • Use the Visual Studio 2012 Add SQL Server icon

NOTE: The dialogue, by which the connection is facilitated, looks different.  Visual Studio 2012 uses the same connection as is used by SQL Server Management Studio.

Once a connection is made to a database, it is listed in the [SQL] Server [Object] Explorer tab, and can consequently be accessed via Visual Studio.

The functionality, hanging off the navigable database objects-tree, is limited to specific database operations relative to the meaningfulness of the node.  Each node in the tree, for example the Databases or Tables nodes, have different menu options onthe right-click menu.  It makes sense when right-clicking a database node that the New Query option would be presented, as opposed to right-clicking a table name where Data View, Code View, and the likes, are presented.

Visual Studio 2012 SQL Server Object Explorer Query

Visual Studio 2012 SQL Server Object Explorer Query

A database query, just like using SQL Server Management Studio, can be created from the SQL Server Object Explorer’s connection to the database, by selecting the appropriate node, viz. SQL Server or Database, using the right-click menu: New Query… to create a query tab.  This query is a bona fide database query, without any context to an object residing in a project, it could perform any database action.

Doing the same in Visual Studio 2010, having the latest version of data tools installed, produces different results.  The New Query option launches a designer view (something like that of Access, or the Database Diagram option) from which to select tables that are to be included in the TSQL query, and restricts, or boxes the developer into a template ability.

Visual Studio 2010 Data Connections Query

Visual Studio 2010 Data Connections Query

Database Diagrams are not present in the Visual Studio’s SQL Server database hierarchy tree, any-more [6], with 2012.  It can still be found in Visual Studio 2010, but only used on databases up to SQL Server 2008.  This is a bit sad, because in my humble opinion, making Visual Studio a one-stop-database-design-shop fails here, because one would really benefit from being able to model a database in the IDE, along with the other functionality.  There is also no modelling option under the Architecture tool-bar option, which leaves SQL Server Management Studio [7] and Visio, as readily available Microsoft modelling tools.

The New Query, accessed from the right-click menu, on a table node (Visual Studio 2010) also launches the designer for a  selection of tables to participate in the query.  Visual Studio 2012 provides: View Code, Designer and Data options, but no New Query.  It does not enable the data-connection tool-bar, showing the database to which the connection points, but implicitly connects to the database to which the object belongs.  This makes sense, because the table is situated within a database, so it goes without saying, but boxes the developer into a view state, when accessing the database from this angle.  But, don’t lament, because when using the added Script-As option, the table’s DDL TSQL is scripted to a new query window (an explicit connection), with the associated tool-bar indicating the database connection, and execute options.  Visual Studio 2010 only provides the boxed-in (implicit) query window for database objects.

NOTE:  To execute an open query, where the Execute icon is presented in a tool-bar (explicitly connected), the short-cut commands and icons are different between versions.

  • Visual Studio 2010: Visual Studio 2010 Query Execute Icon or the short-cut code Ctrl+R
  • Visual Studio 2012: Visual Studio 2012 Query Execute Icon or the short-cut code Ctrl+Shift+E

The Visual Studio 2012 icon is similar to the familiar: Start Debugging (F5) of 2010, which is Execute in SQL Server Management Studio (haven’t we all done that?) and has been changed with a drop-down listing Execute or Execute with debugger (Alt+F5), in 2012.

NOTE: Ctrl-R does not minimise the results-pane, as in SQL Server Management Studio.  Use the key-combination Ctrl+D, R, lifting finger off the Ctrl, after pressing D, to press R.  This however keeps the results pane hidden on next execution, and is brought up again, by repeating the command, hum-ho.

Visual Studio 2012 Query Execution Options

Visual Studio 2012 Query Execution Options

The Programmability (Stored Procedures and Functions) menu option in Visual Studio 2012, provides the Script-As and View Code options.  The latter, by virtue of the word View, provides a View only (implicit connection), i.e. it does not connect to the database for manipulation.  The former Script-As (to new query window) connects to the database for manipulation (explicit connection), i.e. it provides the ability to CREATE the object, i.e. execute the DDL on the database.  In Visual Studio 2010, stored procedures can’t be executed or scripted in the same manner, but only an Add option is provided for a new object.

Using Database Project Objects, connected to a Database

Selecting objects from the Solution Explorer (Visual Studio 2012) opens up the object’s code and design-view, in a non-connected (implicit) state.  However, the project is bound to a database, as specified in the Project Properties tab >> Target Connection String, i.e. implicitly.  When a project database object is displayed, a connection to this database is achieved by right-clicking the code window and selecting: Connection > Connect.  To consequently run the code, on the database, from the same menu, select: Execute, or use the short-cut keys: Ctrl+Shift+E.  The same functionality exists for Visual Studio 2010’s Database Tools.

For a more detailed overview, on working with Visual Studio 2012 Database Projects, view the article: Visual Studio 2012 Database Project (SSDT): Working with a Database Project.

Citations

No. Author Web Site Name URL Date Authored Date Cited
 1.   Kathleen Richards  Visual Studio Magazine  http://visualstudiomagazine.com/articles/2012/11/26/visual-studio-2012-update-released.aspx  26 November2012  07 January2013
 2.  Microsoft  MSDN  http://www.microsoft.com/visualstudio/eng/downloads#d-visual-studio-2012-update  November 2012  07 January2013
 3.  Microsoft  Download Center  http://www.microsoft.com/en-us/download/details.aspx?id=35575&utm_source=MyTechMantra.com  November 2012  07 January2013
 4. sqlexpress SQL Server Experess WebLog http://blogs.msdn.com/b/sqlexpress/archive/2011/07/12/introducing-localdb-a-better-sql-express.aspx 12 July 2011 07 January2013
 5. Microsoft Data DeveloperCenter http://msdn.microsoft.com/en-US/data/tools.aspx December 2012 07 January2013
 6. MSDN Forum MSDN:Visual Studio 11 Database Diagrams http://social.msdn.microsoft.com/Forums/en-US/vstsdb/thread/34154fdf-d81c-4d58-abf4-c27b157dd420 01 April 2012 07 January2013
 7. MSDN MSDN:Visual Database Tool Designers http://msdn.microsoft.com/en-us/library/ms190415.aspx 2012 07 January2013

Last updated: 07 January 2013 16h50