Prefacio

I've finally come full circle. I initially intended to write about a pre-existing wizard that I rebuilt before I found out about xaml controls. Ignore the irony about writing a series on using xaml controls and giving the final how-to article using the old controls but I did build the wizard(s) before I discovered the system; so there's that. Also, as I mentioned at the end of the first article, there's a sample tutorial wizard provided by Sitecore, built using the new xaml controls which can be used if you're up to converting it.

Since I began, I've built three more wizards for myself. In the old system. I know, I know. They were to add/remove sites from the system and add/remove extranet security for any given site. This left me with a series of resuable base classes and xml controls. These classes form the backbone that handles gathering data from each field, displaying progress information to the user, and processing the captured information for one reason or another. The intention is to make it easier to customize the forms, messaging and processing for each wizard. In the following example I'm going to implement a sample wizard with two pages demonstrating populating drop trees, drop downs, multi selectors, a checkbox and handling form validity. The final page shows a summary of the information before you process it. Then the process page uses the long running process code written by Alistair Deney's to execute your code in it's own thread so that the page can be updated, giving good feedback to the user. By breaking your task into sections and updating the message for each section it's a pretty slick tool.  

WIZARD!

For anyone who's unfamiliar with what a wizard is or just not sure what I'm referring to, here's a picture I'm sure you'll be familiar with:

To begin, you'll need to setup a link for an application. The dimensions should be: height-545 and width-500. Next create a "Wizards" folder under the /sitecore modules/shell/ folder and a "SampleWizard" folder beneath that.

Xml Controls

Now we need to begin by creating files for the xml control. To start we'll need the xml control that the application points to. Create the file: 

/sitecore modules/shell/Wizards/SampleWizard/SampleWizard.xml 

Copy the code following code into that file. You should note that the code beside references the WizardCore class. It also uses a WizardFormFirstPage, which is part of the Sitecore system and then two custom pages of type: PageOne and PageTwo. These will be defined next. The next two pages are WizardFormPages also a type provided by Sitecore and they are used for the displaying summary and processing information. The final page is a Sitecore provided WizardFormLastPage that shows when the wizard completes. Here you have the ability to modify some of the messaging in the application. Take note you should update the namespaces and assembly references beginning with "SampleLib" in all the following code with your own class library namespace.

<?xml version="1.0" encoding="utf-8" ?>
<control xmlns:def="Definition" xmlns:content="http://www.sitecore.net/content">
	<SampleWizard>
		<WizardForm CodeBeside="SampleLib.Wizards.SampleWizard.WizardCore,SampleLib">
			<Stylesheet Src="/sitecore modules/shell/Wizards/SampleWizard/css/wizard.css"/>

			<WizardFormFirstPage ID="FirstPage" Icon="Applications/48x48/magic-wand.png">
				<Border Class="scWizardWelcomeTitle">
					<Literal Text="Control Wizard"/>
				</Border>
				<Literal Text="This wizard is an example of a control based Wizard."/>
			</WizardFormFirstPage>

			<PageOne ID="FirstCustomPage"
				PageName="First Page"
				Header="First Page Header"
				Text="Some instructions."
				Icon="Software/32x32/text_code.png"/>

			<PageTwo ID="SecondCustomPage"
				PageName="Second Page"
				Header="Second Page Header."
				Text="Some instructions."
				Icon="Applications/32x32/gears.png"/>

			<WizardFormPage ID="SummaryPage"
				Header="Summary"
				Text="Please confirm your choices before continuing."
				Icon="Applications/48x48/magic-wand.png">
				<Scrollbox Border="none" Background="transparent">
					<Groupbox ID="ChoicesPanel" Header="Your Configuration Choices">
						<Border Padding="4" >
							<Literal ID="Choices" Text="You have selected the following settings:"/>
						</Border>
					</Groupbox>
				</Scrollbox>
			</WizardFormPage>

			<WizardFormPage ID="ProcessPage"
				Header="Building Site"
				Text="Please wait while the site is being created and configured"
				Icon="People/32x32/Box_Software.png">
				<WizardFormIndent>
					<Edit ID="HandleId" Hidden="True"/>
					<GridPanel ID="ProcessDetails" CellPadding="10">
						<Groupbox Header="Step 1 Description">
							<Border>
								<Literal ID="step1Message" Text=" "/>
							</Border>
							<Border>
								<Literal ID="step1Status"/>
							</Border>
						</Groupbox>
						<Groupbox Header="Step 2 Description">
							<Border>
								<Literal ID="step2Message" Text=" "/>
							</Border>
							<Border>
								<Literal ID="step2Status"/>
							</Border>
						</Groupbox>
					</GridPanel>
				</WizardFormIndent>
			</WizardFormPage>

			<WizardFormLastPage ID="LastPage" Icon="Applications/48x48/magic-wand.png">
				<Scrollbox Border="none" Background="transparent">
					<Border Padding="4">
						<Literal ID="FinalMessage" Text="The wizard has completed."/>
					</Border>
				</Scrollbox>
			</WizardFormLastPage>
		</WizardForm>
	</SampleWizard>
</control>

I'll show the two custom pages shortly, but here's a screenshot of the "SummaryPage", "ProcessPage" and "LastPage" included in this control:

Next we'll need some css for the frame and form. It's referenced at the top of the previously defined xml control. You'll want to create:

/sitecore modules/shell/Wizards/SampleWizard/css/wizard.css

fieldset { margin: 10px 10px; }
	fieldset td { vertical-align:top; }
fieldset tr { margin-bottom:7px; display:block; }
fieldset label { width:140px; display:block; text-align:right; margin-right:10px; line-height:20px; }
fieldset input { width:200px; }
fieldset select { width:305px; }
fieldset input.checkbox { width:auto; }
table#Settings > tbody > tr,
table#Language > tbody > tr,
table#Security > tbody > tr { margin-bottom:10px;}
.scWizardHeader { height:64px; }
span.asterisk { color:#ff0000; display:inline-block; margin-right:3px; }
span.value { color:#ff0000; }
.scWizardText { margin-left:10px !important; }
.scComboboxEdit { width: 280px; }
	fieldset .scCombobox tr { margin-bottom: 0px; }
.ErrorMessage { padding:4px; display:block; line-height:17px; }
#ProcessDetails { text-align: center; width: 425px; }
	#ProcessDetails td { padding:5px; }
	#ProcessDetails div { display:inline-block; margin-left:20px; vertical-align:middle; width:162px; }

Then we'll need the xml controls for each individual page referenced on the main xml control. Each page will reference it's own class file. 

On the first page I'm going to demonstrate a TreePicker and a ComboBox. Both are types drop downs but each is unique. Since the TreePicker is displaying a part of the content tree, you need to set a DataContext item for it. You also must create a DataContext item in the xml for this to work. The rest of the settings for the DataContext will be set in the class. The ComboBox will be populated with descendents of the Home item in the content tree in the class file. Create the file:

/sitecore modules/shell/Wizards/SampleWizard/Pages/PageOne.xml

<?xml version="1.0" encoding="utf-8" ?>
<control xmlns:def="Definition" xmlns:content="http://www.sitecore.net/content">
	<PageOne def:inherits="SampleLib.Wizards.SampleWizard.Pages.PageOne,SampleLib">
		<GridPanel Width="100%" Style="display:none">
			<WizardPageHeader GridPanel.Class="scWizardHeader" Header="$Header" Text="$Text" Icon="$Icon"/>
			<WizardPageDivider/>

			<DataContext ID="ExampleDC"/>

			<Groupbox Header="Tree Picker Example">
				<GridPanel Columns="2">
					<Label For="TreeExample"><span class="asterisk">*</span>Tree Picker:</Label>
					<TreePicker ID="TreeExample" DataContext="ExampleDC" ToolTip="This shows how to setup the TreePicker."/>
				</GridPanel>
			</Groupbox>

			<Groupbox Header="Combobox Example">
				<GridPanel Columns="2">
					<Label For="ComboboxExample"><span class="asterisk">*</span>Select an item:</Label>
					<Combobox ID="ComboboxExample" ToolTip="This shows how to setup a Combobox." />
				</GridPanel>
			</Groupbox>
		</GridPanel>
	</PageOne>
</control>

Here's a screenshot of what the first page looks like:

The second page will also show a multiselect list. I'll populate this will language items from Sitecore in the class file. The second part of the form has a checkbox and an error message that identifies how to move forward through the form. The checkbox will be required to be checked to process the form. File:

/sitecore modules/shell/Wizards/SampleWizard/Pages/PageTwo.xml

<?xml version="1.0" encoding="utf-8" ?>
<control xmlns:def="Definition" xmlns:content="http://www.sitecore.net/content">
	<PageTwo def:inherits="SampleLib.Wizards.SampleWizard.Pages.PageTwo,SampleLib">
		<GridPanel Width="100%" Style="display:none">
			<WizardPageHeader GridPanel.Class="scWizardHeader" Header="$Header" Text="$Text" Icon="$Icon"/>
			<WizardPageDivider/>

			<Groupbox Header="Listbox Example">
				<GridPanel Columns="2">
					<Label For="ListboxExample"><span class="asterisk">*</span>Choose Item: </Label>
					<Listbox ID="ListboxExample" Multiple="false" size="10" ToolTip="This is an example of a Listbox."/>
				</GridPanel>
			</Groupbox>

			<Groupbox Header="Checkbox Example">
				<GridPanel Columns="2">
					<Literal GridPanel.ColSpan="2" ID="PageTwoErrorMessage" Class="ErrorMessage" Visible="false" Style="color:red" />

					<Label For="CheckboxExample">Check or not: </Label>
					<Checkbox ID="CheckboxExample" Class="checkbox" ToolTip="This is an example of a Checkbox."/>
				</GridPanel>
			</Groupbox>
		</GridPanel>
	</PageTwo>
</control>

Here's a screenshot of the second page:

There are also header chunks that are broken out into two more files. They live just under the Wizards folder because they are resuable to all wizards:

/sitecore modules/shell/Wizards/WizardPageDivider.xml

<?xml version="1.0" encoding="utf-8" ?>
<control xmlns:def="Definition" xmlns:content="http://www.sitecore.net/content">
	<WizardPageDivider>
		<GridPanel Width="100%">
			<Space GridPanel.Class="scBottomEdge"/>
			<Space GridPanel.Class="scTopEdge"/>
		</GridPanel>
	</WizardPageDivider>
</control>

/sitecore modules/shell/Wizards/WizardPageHeader.xml

<?xml version="1.0" encoding="utf-8" ?>
<control xmlns:def="Definition" xmlns:content="http://www.sitecore.net/content">
	<WizardPageHeader>
		<GridPanel Columns="2">
			<Border GridPanel.Width="100%" GridPanel.VAlign="top">
				<Border Class="scWizardTitle">
					<Literal Text="$Header"/>
				</Border>
				<Border Class="scWizardText">
					<Literal Text="$Text"/>
				</Border>
			</Border>
			<ThemedImage Src="$Icon" Width="32" Height="32" Margin="0px 8px 0px 0px"/>
		</GridPanel>
	</WizardPageHeader>
</control>

That's all the xml control files that make up the wizard structure. You're folder should look like this:

Config Settings

We now need to have the backing classes that run the show but first we should setup a reference to the library we'll be using in an include config file: 

/App_Config/includes/Sample.config

<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
	<sitecore>
		<ui>
			<references>
				<reference comment="SampleLib">/bin/SampleLib.dll</reference>
			</references>
		</ui>
	</sitecore>
</configuration>

Base Classes

Next we'll start creating the reusable base classes. To match the sitecore modules folders, you should create a "Wizards" folder and a "SampleWizard" folder beneath it. The AbstractWizardCore is the base class of the application itself. Here is where the movement between pages is managed, the messaging and the thread that processes the final job/task or whatever it is you're doing. You'll want to reference System.Configuration, System.Web, Sitecore.Kernel and Sitecore.Client from your library. 

/SampleLib/Wizards/AbstractWizardCore.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Sitecore;
using Sitecore.Data.Managers;
using Sitecore.Jobs;
using Sitecore.Shell.Framework;
using Sitecore.Web.UI.HtmlControls;
using Sitecore.Web.UI.Pages;
using Sitecore.Web.UI.Sheer;
using HtmlLiteral = Sitecore.Web.UI.HtmlControls.Literal;

namespace SampleLib.Wizards {
	public abstract class AbstractWizardCore : WizardForm {

		#region Pages
		public static readonly string ProcessPage = "ProcessPage";
		public static readonly string SummaryPage = "SummaryPage";
		public static readonly string LastPage = "LastPage";
		#endregion Pages

		#region Controls
		protected HtmlLiteral Choices;
		protected HtmlLiteral FinalMessage;
		//process build id
		protected Edit HandleId;
		#endregion Controls

		#region Settings

		protected virtual int RefreshTime { get { return 200; } }

		protected abstract int TotalSteps { get; }

		protected abstract string ExecuteBtnText { get; }

		protected abstract string JobName { get; }

		#endregion Settings

		#region Control Groupings

		protected abstract List<HtmlLiteral> MessageFields { get; }

		protected abstract List<HtmlLiteral> StatusImages { get; }

		protected Dictionary<StatusType, ImageSet> StatusTypes {
			get {
				return new Dictionary<StatusType, ImageSet>() {
					{ StatusType.progress, new ImageSet(){ Src="Images/Progress.gif", Height=17, Width=94 } },
					{ StatusType.failed, new ImageSet(){ Src="Applications/32x32/delete.png", Height=32, Width=32 } },
					{ StatusType.passed, new ImageSet(){ Src="Applications/32x32/check.png", Height=32, Width=32 } },
					{ StatusType.queued, new ImageSet(){ Src="People/32x32/stopwatch.png", Height=32, Width=32 } }
				};
			}
		}

		public enum StatusType { progress, failed, passed, queued };

		protected class ImageSet {
			public string Src;
			public int Height;
			public int Width;
		}

		#endregion Control Groupings

		#region Properties

		protected Job BuildJob {
			get {
				Handle handle = Handle.Parse(HandleId.Value);
				return JobManager.GetJob(handle);
			}
		}

		protected IEnumerable<BasePage> SiteBuilderPages {
			get {
				// for each page id, find the control
				var q = from string val in this.Pages select Context.ClientPage.FindControl(val);
				// return only sitebuilder pages, cast to sitebuilder pages.
				return q.OfType<BasePage>().Cast<BasePage>();
			}
		}

		protected BasePage CurrentSiteBuilderPage {
			get {
				var ret = Context.ClientPage.FindControl(this.Active) as BasePage;
				return ret;
			}
		}

		#endregion Properties

		#region Page Changing

		protected virtual bool HasCustomPageChangingEvent(string page, string newpage) { return false; }

		protected override void OnNext(object sender, EventArgs formEventArgs) {
			if (null != CurrentSiteBuilderPage) {
				if (CurrentSiteBuilderPage.IsValid) {
					base.OnNext(sender, formEventArgs);
				}
			} else {
				base.OnNext(sender, formEventArgs);
			}
		}

		protected override bool ActivePageChanging(string page, ref string newpage) {

			NextButton.Header = "Next";
			if(HasCustomPageChangingEvent(page, newpage)){
				return true;
			} else if (newpage == SummaryPage) {
				NextButton.Header = ExecuteBtnText;
				// invokes an aggegate function on each sitebuilder page using a new string builder
				// as the aggregate object to collect the output.
				StringBuilder sb = SiteBuilderPages.Aggregate(new StringBuilder(), (acc, aPage) => {
					acc.AppendFormat(@"<h4>{0}</h4>", aPage.PageName);
					acc.Append(@"<ul>");
					foreach (string val in aPage.DataSummary) {
						acc.AppendFormat("<li>{0}</li>", val);
					}
					acc.Append(@"</ul>");
					return acc;
				});
				Choices.Text = sb.ToString();
			} else if (newpage == ProcessPage) {

				// performs an aggregation function on each sitebuilder page using a new dictionary as the aggregate object.
				// Collects the data from all pages and builds the requested site.
				Dictionary<string, object> data = SiteBuilderPages.Aggregate(new Dictionary<string, object>(), (d, aPage) => d.Merge(aPage.DataDictionary, false));

				//disable the buttons and start the long running process
				NextButton.Visible = false;
				BackButton.Visible = false;

				for (int i = 0; i < TotalSteps; i++) {
					SetStatus((i+1), StatusType.queued, " ");
				}

				Job job = JobManager.Start(new JobOptions(
				  JobName,
				  "Wizard Tools",
				  Sitecore.Context.Site.Name,
				  this,
				  "ProcessBuild",
				  new object[] { data }));
				job.Status.Total = TotalSteps;

				HandleId.Value = job.Handle.ToString();
				SheerResponse.Timer("CheckBuildStatus", RefreshTime);
			}

			return true;
		}

		#endregion Page Changing

		#region Building

		protected abstract AbstractLongRunningJob GetJobObject(Job j);

		protected void ProcessBuild(Dictionary<string, object> data) {
			AbstractLongRunningJob blrj = GetJobObject(BuildJob);
			blrj.Execute(data);
		}

		protected void CheckBuildStatus() {
			try {
				//get message info
				int last = BuildJob.Status.Messages.Count - 1;
				string message = (last > -1) ? BuildJob.Status.Messages[last] : "no messages";

				//set status message
				int step = (int)BuildJob.Status.Processed;
				if (step > 0 && step <= BuildJob.Status.Total) {
					//set last step as finished as long as there is a last step
					if (step > 1) {
						SetStatus(step - 1, StatusType.passed, "Completed.");
					}
					//set current step as in progress
					SetStatus(step, StatusType.progress, message);
				}

				if (!BuildJob.IsDone) {
					SheerResponse.Timer("CheckBuildStatus", RefreshTime);
				} else {
					//on finish the build job adds an additional message so grab the 2nd to last message if it's the last one.
					message = (last > 0) ? BuildJob.Status.Messages[last - 1] : BuildJob.Status.Messages[last];
					BuildComplete((BuildJob.Status.Failed) ? StatusType.failed : StatusType.passed, (BuildJob.Status.Failed) ? "Failed" : "Passed", (message.Length > 0) ? message : "The Site Builder Wizard has completed.");
				}
			} catch (Exception ex) {
				BuildComplete(StatusType.failed, "Check Build Status threw an exception", ex.ToString());
			}
		}

		protected void BuildComplete(StatusType t, string statusText, string message) {
			//set last status
			int step = (int)BuildJob.Status.Processed;
			SetStatus(step, t, "Completed.");

			//set the last message and button states
			ImageSet p = StatusTypes[t];
			FinalMessage.Text = string.Format("Build Completed.  {0}<br/><br/>Status: {1}<br/><br/>Message:<br/><br/>{2}", ThemeManager.GetImage(p.Src, p.Width, p.Height), statusText, message);

			//finished. go to the next page
			this.Next();
		}

		protected void SetStatus(int step, StatusType t, string message) {
			int pos = (step > 0) ? step - 1 : 0;
			HtmlLiteral i = StatusImages[pos];
			HtmlLiteral m = MessageFields[pos];
			ImageSet p = StatusTypes[t];
			i.Text = ThemeManager.GetImage(p.Src, p.Width, p.Height);
			m.Text = message;
		}

		#endregion Building

		#region Cancel Wizard

		protected override void OnCancel(object sender, EventArgs formEventArgs) {
			if (this.Active == LastPage) {
				Windows.Close();
			} else {
				Context.ClientPage.Start(this, "Confirmation");
			}
		}

		public new void Confirmation(ClientPipelineArgs args) {
			if (null == args.Result) {
				Context.ClientPage.ClientResponse.Confirm("Are you sure you want to close the wizard?");
				args.Suspend(true);
			} else if (args.Result == "yes") {
				Windows.Close();
			}
		}

		#endregion Cancel Wizard
	}
}

Each page also uses a BasePage class that provides access to the databases, does validation checks, provides summary and data information to its parent wizard core.

/SampleLib/Wizards/BasePage.cs

using System;
using System.Collections.Generic;
using System.Linq;
using Sitecore.Data;
using Sitecore.Web.UI.Pages;

namespace SampleLib.Wizards {

	public class BasePage : WizardDialogBaseXmlControl {

		private Database _mdb;
		public Database MasterDB {
			get {
				if (_mdb == null)
					_mdb = Sitecore.Configuration.Factory.GetDatabase("master");
				return _mdb;
			}
		}

		private Database _wdb;
		public Database WebDB {
			get {
				if (_wdb == null)
					_wdb = Sitecore.Configuration.Factory.GetDatabase("web");
				return _wdb;
			}
		}

		public string PageName { get; set; }

		public virtual bool IsValid { get { return true; } }

		public virtual IEnumerable<string> DataSummary {
			get {
				return from val in DataDictionary select FormatSummary(val.Key, val.Value.ToString());
			}
		}

		protected virtual string FormatSummary(string key, string value){
			return string.Format(@"{0}: <span class='value'>{1}</span>", key, value);
		}

		public virtual IEnumerable<KeyValuePair<string, object>> DataDictionary {
			get {
				yield break;
			}
		}

		protected override void OnLoad(EventArgs e) {

			if (!Sitecore.Context.ClientPage.IsEvent) {
				InitializeControl();
			}
		}

		protected virtual void InitializeControl() { }
	}
}

The AbstractLongRunningJob is the base class for the final processing functionality you'll be doing. It provides database access, user messaging and a transactional cleanup capability should something go wrong.

/SampleLib/Wizards/AbstractLongRunningJob.cs 

using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Jobs;
using Sitecore.Security.Accounts;
using Sitecore.SecurityModel;

namespace SampleLib.Wizards {
	public abstract class AbstractLongRunningJob {

		protected List<object> CleanupList;
		protected Database MasterDB;
		protected Database WebDB;
		protected Job BuildJob;
		protected Dictionary<string, object> InputData;

		public AbstractLongRunningJob(Job job) {
			CleanupList = new List<object>();
			BuildJob = job;
			MasterDB = Sitecore.Configuration.Factory.GetDatabase("master");
			WebDB = Sitecore.Configuration.Factory.GetDatabase("web");
		}

		#region Messaging

		protected int LangCur;
		protected int LangTotal;
		protected int ItemCur;
		protected int ItemTotal;

		protected void SetStatus(int processed) {
			SetStatus(processed, string.Empty);
		}
		protected void SetStatus(string message) {
			SetStatus(-1, message);
		}
		protected void SetStatus(int processed, string message) {
			if(processed > -1)
				BuildJob.Status.Processed = processed;
			if(!string.IsNullOrEmpty(message))
				BuildJob.Status.Messages.Add(message);
		}

		#endregion Messaging

		#region Execute

		public void Execute(Dictionary<string, object> data) {

			BuildJob = Sitecore.Context.Job;
			SetStatus(0, "Starting Process.");
			InputData = data;
			CleanupList.Clear();

			try {
				using (new SecurityDisabler()) {
					CoreExecute(data);
				}
				BuildJob.Status.Messages.Add("Finished Successfully");
			} catch (Exception ex) {
				StringBuilder sb = new StringBuilder(ex.ToString());
				CleanupOnFail(ref sb);
				BuildJob.Status.Failed = true;
				BuildJob.Status.Messages.Add(string.Format("The wizard was unable to complete because of the following error(s): <br/>{0}", sb.ToString()));
			}
		}

		public abstract void CoreExecute(Dictionary<string, object> data);

		#endregion

		#region Transactional Methods

		protected void CleanupOnFail(ref StringBuilder message) {
			using (new SecurityDisabler()) {
				foreach (var val in CleanupList) {
					try {
						if (val is Item) {
							(val as Item).Delete();
						} else if (val is DirectoryInfo) {
							(val as DirectoryInfo).Delete(true);
						} else if (val is Role) {
							System.Web.Security.Roles.DeleteRole((val as Role).Name);
						}
					} catch (System.Exception ex) {
						message.AppendLine();
						message.AppendFormat("Failed to cleanup [{0}] because of --> {1}", val.ToString(), ex.Message);
					}
				}
			}
		}

		#endregion Transactional Methods
	}
}

There are also a few extension methods that are used in other classes.

/SampleLib/Wizards/WizardExtensions.cs

using System.Collections.Generic;
using System.Linq;

namespace SampleLib.Wizards {
	public static class WizardExtensions {

		public static T Get<T>(this Dictionary<string, object> data, string parameter) {
			object ret = null;
			if (data.TryGetValue(parameter, out ret) && ret is T)
				return (T)ret;
			else
				return default(T);
		}
	}

	public static class DictionaryExtensions {
		public static Dictionary<K, V> Merge<K, V>(this Dictionary<K, V> target, IEnumerable<KeyValuePair<K, V>> source, bool overwrite) {
			source.Aggregate(target, (acc, kvp) => {
				if (!acc.ContainsKey(kvp.Key)) {
					acc.Add(kvp.Key, kvp.Value);
				} else if (overwrite) {
					acc[kvp.Key] = kvp.Value;
				}
				return acc;
			});
			return target;
		}

	}
}

Sample Wizard

Now we get to the heart of the sample wizard. The WizardCore class is the implementation of the AbstractWizardCore which is what the application runs on. When you create this class you're forced to provide implementation information such as the number of steps you want to break your task into the text for the button, references to front end messaging controls and an implementation of an AbstractLongRunningJob class. The number provided for the TotalSteps should coincide with the number of pairs of message and status HtmlLiteral controls as well as their references in the MessageFields and StatusImages properties. The refer to the controls on the WizardCore.xml files so if you add or remove any, you should update that file as well. You can fit four comfortably, possibly five. 

/SampleLib/Wizards/SampleWizard/WizardCore.cs

using System.Collections.Generic;
using Sitecore.Jobs;
using HtmlLiteral = Sitecore.Web.UI.HtmlControls.Literal;

namespace SampleLib.Wizards.SampleWizard {
	public class WizardCore : AbstractWizardCore {

		#region Pages
		public static readonly string PageOne = "FirstCustomPage";
		public static readonly string PageTwo = "SecondCustomPage";
		#endregion Pages

		#region Controls
		//process steps
		protected HtmlLiteral step1Message;
		protected HtmlLiteral step1Status;
		protected HtmlLiteral step2Message;
		protected HtmlLiteral step2Status;
		#endregion Controls

		#region Settings

		protected override int TotalSteps { get { return 2; } }

		protected override string ExecuteBtnText { get { return "Start Job"; } }

		protected override string JobName { get { return "Control Wizard Example"; } }

		#endregion Settings

		#region Control Groupings

		protected override List<HtmlLiteral> MessageFields {
			get {
				return new List<HtmlLiteral>() { step1Message, step2Message };
			}
		}

		protected override List<HtmlLiteral> StatusImages {
			get {
				return new List<HtmlLiteral>() { step1Status, step2Status };
			}
		}

		#endregion Control Groupings

		#region Page Changing

		protected override bool HasCustomPageChangingEvent(string page, string newpage) {

			if (!newpage.Equals(PageOne))
				return false;

			return true;
		}

		#endregion Page Changing

		#region Building

		protected override AbstractLongRunningJob GetJobObject(Job j) {
			return new SampleJob(j);
		}

		#endregion Building
	}
}

The implementation of the AbstractLongRunningJob we're using, is the SampleJob and will pull references from the data stored previously by each individual page. You can then do work on that data, update the status and change the step you're working. In this case I'm just going to be counting to ten for each step to show you how to use the messaging. Otherwise no real work is actually done in this example.  

/SampleLib/Wizards/SampleWizard/SampleJob.cs

using System.Collections.Generic;
using Sitecore.Data.Items;
using Sitecore.Globalization;
using Sitecore.Jobs;

namespace SampleLib.Wizards.SampleWizard {
	public class SampleJob : AbstractLongRunningJob {

		public SampleJob(Job job) : base(job) { }

		#region Start Build

		public override void CoreExecute(Dictionary<string,object> data) {

			Item TreeItem = InputData.Get<Item>(Constants.Keys.TreeItem);
			Item ComboItem = InputData.Get<Item>(Constants.Keys.ComboItem);
			Language ListItem = InputData.Get<Language>(Constants.Keys.ListItem);
			bool CheckItem = InputData.Get<bool>(Constants.Keys.CheckItem);

			//status
			SetStatus(1, "Step 1.");
			Method();

			//status
			SetStatus(2, "Step 2.");
			Method();
		}

		#endregion Start Build

		#region Build Chunks

		protected void Method() {
			for (int i = 0; i < 10; i++) {
				SetStatus(string.Format("substep {0}", i.ToString()));
				System.Threading.Thread.Sleep(500);
			}
		}

		#endregion Build Chunks
	}
}

There is also a class file to store strings used throughout the wizard. 

/SampleLib/Wizards/SampleWizard/Constants.cs

namespace SampleLib.Wizards.SampleWizard {
	public static class Constants {
		public static class Keys {
			public static readonly string TreeItem = "TreeItem";
			public static readonly string ComboItem = "ComboItem";
			public static readonly string ListItem = "ListItem";
			public static readonly string CheckItem = "CheckItem";
		}

		public static class Paths {
			public static readonly string Home = "/sitecore/content/Home";
		}

		public static class ItemIDs {
			public static readonly string Home = "{110D559F-DEA5-42EA-9C1C-8A5DF7E70EF9}";
		}
	}
}

The last two pieces of the puzzle are the implementations of the individual pages. For each, setup references to the front end controls. Then build out the DataSummary and DataDictionary properties that provide the data to the larger wizard application and long running job. Both pages will use the InitializeControl method to setup the fields.

The DataContext on PageOne requires some special configuration to get it working. PageTwo overrides the IsValid property to do form validation and control page movement.

/SampleLib/Wizards/SampleWizard/Pages/PageOne.cs

using System.Collections.Generic;
using System.Linq;
using Sitecore.Data.Items;
using Sitecore.Web.UI.HtmlControls;

namespace SampleLib.Wizards.SampleWizard.Pages {
	public class PageOne : BasePage {

		#region Controls
		protected DataContext ExampleDC;
		protected TreePicker TreeExample;
		protected Combobox ComboboxExample;
		#endregion Controls

		#region Properties

		public override IEnumerable<string> DataSummary {
			get {
				return new List<string> {
					FormatSummary(Constants.Keys.TreeItem, MasterDB.GetItem(TreeExample.Value).Name),
					FormatSummary(Constants.Keys.ComboItem, MasterDB.GetItem(ComboboxExample.Value).Name)
				};
			}
		}

		public override IEnumerable<KeyValuePair<string, object>> DataDictionary {
			get {
				yield return new KeyValuePair<string, object>(Constants.Keys.TreeItem, MasterDB.GetItem(TreeExample.Value));
				yield return new KeyValuePair<string, object>(Constants.Keys.ComboItem, MasterDB.GetItem(ComboboxExample.Value));
			}
		}

		#endregion Properties

		#region Page Load

		protected override void InitializeControl() {

			//setup the datacontext for the treepicker
			ExampleDC.GetFromQueryString();
			ExampleDC.Root = Constants.Paths.Home;
			ExampleDC.Folder = Constants.ItemIDs.Home;

			//setup drop downs
			Item pFolder = MasterDB.GetItem(Constants.Paths.Home);
			if (pFolder == null)
				return;
			IEnumerable<Item> pages =
				from val in pFolder.Axes.GetDescendants()
				orderby val.Name
				select val;

			foreach (Item val in pages) {
				ListItem li1 = new ListItem() { ID = Control.GetUniqueID("I"), Header = val.DisplayName, Value = val.ID.ToString(), Selected = false };
				ListItem li2 = new ListItem() { ID = Control.GetUniqueID("I"), Header = val.DisplayName, Value = val.ID.ToString(), Selected = false };

				Sitecore.Context.ClientPage.AddControl(TreeExample, li1);
				Sitecore.Context.ClientPage.AddControl(ComboboxExample, li2);
			}
		}

		#endregion Page Load
	}
}

/SampleLib/Wizards/SampleWizard/Pages/PageTwo.cs

using System.Collections.Generic;
using System.Linq;
using Sitecore.Globalization;
using Sitecore.Web.UI.HtmlControls;

namespace SampleLib.Wizards.SampleWizard.Pages {
	public class PageTwo : BasePage {

		#region Page
		protected Literal PageTwoErrorMessage;
		protected Listbox ListboxExample;
		protected Checkbox CheckboxExample;
		#endregion Page;

		#region Properties

		public override bool IsValid {
			get {
				bool valid = CheckboxExample.Checked;
				if (!valid) {
					PageTwoErrorMessage.Visible = true;
					PageTwoErrorMessage.Text = "You should check the checkbox to proceed";
					Sitecore.Context.ClientPage.ClientResponse.SetOuterHtml(PageTwoErrorMessage.ID, PageTwoErrorMessage);
				}
				return valid;
			}
		}

		public override IEnumerable<string> DataSummary {
			get {
				yield return FormatSummary(Constants.Keys.ListItem, ListboxExample.Value);
				yield return FormatSummary(Constants.Keys.CheckItem, CheckboxExample.Checked.ToString());
			}
		}

		public override IEnumerable<KeyValuePair<string, object>> DataDictionary {
			get {
				Language selectedLang = (from val in MasterDB.Languages
										where val.Name.Equals(ListboxExample.Value)
										select val).First();
				yield return new KeyValuePair<string, object>(Constants.Keys.ListItem, selectedLang);
				yield return new KeyValuePair<string, object>(Constants.Keys.CheckItem, CheckboxExample.Checked);
			}
		}

		#endregion Properties

		#region Page Load

		protected override void InitializeControl() {

			IEnumerable<Language> langs = from val in MasterDB.Languages orderby val.Name select val;

			foreach (Language l in langs) {
				ListItem li = new ListItem() {
					ID = Control.GetUniqueID("I"),
					Header = l.CultureInfo.DisplayName,
					Value = l.Name,
					Selected = (l.Name == Sitecore.Context.Language.Name) };
				Sitecore.Context.ClientPage.AddControl(ListboxExample, li);
			}
		}

		#endregion Page Load
	}
}

You're class library should look like this:

Run it

Now with all the xml controls and backing class files setup, build the project and open the application. Here's the complete series of images that show the progress through the form to completion.

Alright that about does it. Now get out there and automate your little hearts out. Let me know how it goes.

Diatribe

I began working with web testing years ago with the MS Test framework when I got a version of Visual Studio that allowed it's use. I started small and began building simple ping tests. Over time it developed into a tool would run sets of tests daily and fed the results into a blog that I could read every morning. It tought me a lot but eventually I hit too many limitations. License cost being the most obvious but also I couldn't create a custom interface for it. I had to work within Visual Studio and the UI was too blunt. This forced me to finally bite the bullet and start converting my system to NUnit and Watin.

I originally only expected to build a simple web app to run my web tests but it quickly became something much more comprehensive. I decided to incorporate both web(integration) tests and unit tests. Up until now, unit tests were something I've avoided because they were too time/code intensive. For any given piece of code you'll likely need several tests. On the other hand, you can get quite of a bit of mileage from a single web test. Now web testing is not a replacement for unit tests. Really they're complementary. Unit tests will catch all types of things like logical errors, regression errors etc. but web testing will catch context errors. A unit of code may purposefully return an exception and that will pass a unit tests but as useful as a stack trace is, users of your website won't ever think so. That being said, just because you stop an exception from being thrown doesn't mean there isn't an error. So both are needed to support a reliable system.

Shout Outs

I want to give credit where it is due. I spent a lot of time reading about unit testing and testing Sitecore in particular. I found a helpful kickstart article about unit testing Sitecore specifically by Matt Kenny. Another by Alistair Deneys that provides a tool to run your tests. Another article by Dan Solovay detailing a solution to some context issues. A really insightful Wikipedia article on Code Coverage and a great article about the philosophy of unit tests. Apparently there was some wellspring article by Mike Edwards but the page appears to have vanished. 

"TestStar"

So I named the tool "TestStar". I'm clearly a fan of Star Wars. In my mind building this tool was an enormous task that wields a great amount of power and it focuses a lot of scrutiny on any given site so.... yeah, TestStar. You can get it on GitHub and I assure you, it is fully operational. I've also put up a video on YouTube to explain it a little more visually. 

At it's core it's a web application with separate projects for unit and web tests. The intention is to make a companion application to a continuous integration server so that you can copy the binaries and run unit tests, then copy the binaries to your web application and run your web tests. The web app will run these tests through the webpages and when you've got a test configuration you like, there's a form to create a batch file with those settings. This can then be tied up to scheduled tasks and be run hands-free. My end use is to test Sitecore and my associated libraries with it, so I'll go into that towards the end.

Alright now that we got that out of the way let's get down to the nitty gritty.

Framework

I've packaged the NUnit and Watin libraries in with the project. I realize this creates dependencies and will not automatically update NUnit and Watin but I wanted to have the application be functional out of the box. You should be aware of that though. You can update the version on your own and/or fork it and do whatever you like to it.

Core

The NUnitTesting.Core library has set of tools to retrieve test fixtures, test methods and categories from an assembly. The core of the code handles running tests and delegating events and corresponding interfaces which define how to handle test events. The web testing requires a bit of structure to store domain names, domain prefixes, environments etc. so there's entity classes and retrieval methods. The data for web testing is stored in JSON in local files and is reconstituted through deserialization. There's also some configuration information stored in the config files so there's a utility that will retrieve those values as well.

WebTests

The NUnitTesting.WebTests library is where the web tests should be written. This is also a place where you'll likely want to add a fair amount of customization. In my implementation I created a few other base test classes and entities specific for my systems and Sitecore. I've left a few samples to show what's involved in creating a test and how they are displayed on the web application and I'll also explain it in more detail later in this post.

UnitTests

The NUnitTesting.UnitTests library is where unit tests should be written. I could've put all the tests into a single project but I found that separating them made more sense. You'll need to include references to any library you intend to test but I'm sure that's standard fare. Aside from that, you should use categories on either the test fixture or test method to help group the tests. If you intend to test Sitecore, you should read the Matt Kenny article detailing the setup which will help you debug your tests but when you build the library. The binaries (including the ones your testing) will end up in the NUnitTesting.WebApp project. You will still need to add your config files to the NUnitTesting.WebApp project but you won't need to manipulate the App_Config path since it will run just like Sitecore does.

TestLauncher

The NUnitTesting.TestLauncher is a console application that your batch scripts will call to run your tests. You probably won't have to modify it at all but you can debug it if you need to understand why something isn't working or if you're curious about how the chain of events works. There are separate test handlers for unit and web tests because they require different parameters and may output messages differently.

WebApp

The NUnitTesting.WebApp is where everything converges. There are really only three pages: Unit Test page, Web Test page and Web Test Data Manager page. The data manager page is how you'll start creating the information you'll need to run web tests against. This information is stored in JSON locally under the /data folder. Once you have the information, you're then able to run the web tests you've built against that data. The unit test will run against the binaries you have locally in the project.

This is also the intended root of where all the batch scripts will run from and by extension the console app and web/unit tests. There's a /scripts folder where batch scripts are stored and there's even a sample to show how to use it.

Web Testing

To get started with web testing you'll first need to understand a few concepts that the system is built on. These have developed organically over time.

Environments

An environment is largely defined by the domain prefix. In my sphere of influence, many of my sites are duplicated across several environments so that I can test various branches of code independently. I tend to create similar subdomains for each site environment. IE: my local environment has the "http://test." prefix and my integration environment has the "http://integration." prefix. I have around six different environments not all site live support each environment so the site itself stores the site-to-environments relationship information. For unique situations, the environment values, including the domain prefix, can be overridden on the local value stored on the site.

Systems

A system is really a collection of sites. I use it to be able to select multiple sites at one time. It really only is just a name and any given site only belongs to one system. This evolved because I wanted to be able to target a certain test against a group of sites. So instead of having to select every one individually, I can select by the group.

Sites

A site stores information such as root domain, environments, system but also has a key/value store called properties to allow extensibility. I use this to store Sitecore specific information such as site ID and language code. I can then write a subclass of the site class which exposes a property that will retrieve the values from the key/value store.

Tests

Tests are the modular components and should be customized to your specific environment. The project comes with two samples: PingTest and SitemapTest. To build a custom web test, you should extend the BaseWebTest abstract class and decorate the class with the [TestFixture, RequiresSTA] attribute. The BaseWebTest class provides contextual information that allows you to decide what you want the test to do for a given site and environment. The RunTest method is called when the test is run. It is also up to you to update the RequestURL and ResponseStatus properties to tell the event handlers what happened during the test.

Unit Testing

Tests

Unit tests are standard NUnit tests and only require that the class has the [TestFixture] attribute and the test method have the [Test] attribute. You should also set a Category on either the [TestFixture] or [Test] attribute so that you can select groups of tests to run from the web app. For examples of what's available to you when setting up an NUnit test class refer to NUnit.org's documentation.

App Settings

There's a handful of settings that are stored in the web.config and app.config files. This will allow you to configure and customize the system to so that the web pages and batch script creators will use a different library to load and launch tests from and the web test data can be stored in different files. It also helps when debugging the applications to have the values stored locally in the project app.config files. Here's the breakdown of those settings.

EnvironmentsDataFile

This is the file name where the web testing environments will be stored. The file lives under the NUnitTesting.WebApp/data folder.

SystemsDataFile

Similar to the environments file, this is the file that stores the web testing systems.

SitesDataFile

Like the two previous files, this lives in the same folder and stores the web testing site information

DefaultWebTestAssembly

This defines which assembly contains the web tests

DefaultWebTestLauncher

This defines which assembly contains the web test launcher console application.

DefaultUnitTestAssembly

This defines which assembly contains the unit tests

DefaultUnitTestLauncher

This defines which assembly contains the unit test launcher console application.

Batch Scripting

For me the end game was to have the tests that I designed be run from scheduled tasks and continuous integration build events on my server. To that end, both web and unit testing pages have a form to create a batch script from the test configuration selected on the page. This can help you get up and running more quickly especially if you're not familiar with the syntax of batch scripts. There's some caveats with the web testing batch creator. The systems are essentially a grouping of sites but when the script is created the system value won't be used in the startup params. This is because you could select a system and then deselect a single site and the system will still be selected. So to prevent any issues, it just creates a list of the site id's. If you've got the saavy you can replace the list of id's with a single system id manually though.

The syntax of the batch script is specific to what the console application is expecting for input parameters. The unit and web test runner each has a different set of parameters. Here's the breakdown:

Web Tests: [0 - test type] [1 - test assembly] [2 - test name] [3 - environments] [4 - systems (optional)] [5 - sites (optional)]

The environments, systems and sites are all comma separated lists of the id's of their respective objects. The id's can be determined from the manager page where the id is displayed next to each item in the drop down lists. The id's are ordered in a zero-based index. Also although systems and sites are optional, at least one of them should be set.

example:

@echo off
set TestLauncherPath=C:\PathToNUnitTesting\Website\bin\NUnitTesting.TestLauncher.exe

@echo on
"%TestLauncherPath%" "-w" "NUnitTesting.WebTests" "SitemapTest" "3,4" "0" ""

Unit: [0 - test type] [1 - test assembly] [2 - test categories (optional)] [3 - test methods (optional)]

The test categories and test methods are comma separated lists of strings that match the category name or method name from the unit test classes. Both are optional but at least one should be set.

example:

@echo off
set TestLauncherPath=C:\PathToNUnitTesting\Website\bin\NUnitTesting.TestLauncher.exe

@echo on
"%TestLauncherPath%" "-u" "NUnitTesting.UnitTests" "Some Test Category,Another Test Category" "SomeNullTest"

Sitecore Testing

It is possible to test a Sitecore system and it's associated class libraries from this system. It was actually it's intended purpose but I built it to work as a standalone utility for any web/unit tests. There are a handful of configurations you will have to make locally which are well explained by Mathew Kenny but I'll reiterate. Fair warning; this will test your skill level in working with application contexts.

WebApp

For the web application to run the unit tests, you'll need to copy in some information from your existing Sitecore site.

1. You'll need to copy the following sections into your web.config file: Sitecore, appSettings, connectionStrings and configSections.

2. You'll also need to copy in the App_Config folder so there's a local reference to all the Sitecore configurations.

3. Next you'll need to add project references to Sitecore libraries and your site libraries.

4. You'll either need to create a data folder and copy in the license file then update your config to that folder, or update the data file path to an existing folder.

TestLauncher, UnitTests

To enable debugging the class library projects you'll have to setup the project similar to a web application.

1. You'll need to create an App.config file and add the following sections from your web.config: Sitecore, appSettings, connectionStrings and configSections.

2. You'll then need to update all references to "/App_Config" in your App.config file to ".\App_Config" because it needs a relative path to these files.

3. You'll also need to copy in the App_Config folder so there's a local reference to all the Sitecore configurations. You should know that these files all get copied into the bin folder and run from there in case you're having trouble determing where it's context is.

4. Next you'll need to add project references to Sitecore libraries and your site libraries from the WebApp project.

5. You'll either need to create a data folder and copy in the license file then update your config to that folder, or update the data file path to an existing folder.

6. If you're working on WebTests in the TestLauncher you'll need to create a data folder and copy the sites, systems and environments files from the WebApp project

It has been suggested that you can create a patch include config to update all the modified settings. This is true except for one; the connection strings path.

Since the TestLauncher requires a config file to help it run these libraries you'll see in the TestLauncher project's bin folder a NUnitTesting.TestLauncher.exe.config file which copies the app.config settings. You'll need this file in the WebApp project for it to run but, of course, there's an exception; the ".\App_Config" path anywhere within the "Sitecore" section in the config file needs to be set to "..\App_Config". This is because the context of the TestLauncher is partially in the scripts folder where the exe is called from and partially in the bin folder where the exe lives. I've kept a separate copy of the App.config and I copy it into the WebApp on the Post Build Event. Here's the code for that:

copy "$(ProjectDir)App.config.WebApp" "$(SolutionDir)Website\bin\NUnitTesting.TestLauncher.exe.config"

Unit Test Sitecore Context

When you're creating unit tests for Sitecore you'll want to be able to set context data such as the site, language, database, device and whether or not the system is unit testing. I've handled this by setting these manually in the Setup method decorated by the [Setup] attribute. For some tests I needed to have a live site content to test against so I setup a test website in my system. Here's the code I'm using to create that context:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Sitecore.Collections;
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Globalization;
using Sitecore.SecurityModel;
using Sitecore.Sites;
using Sitecore.Web;

namespace NUnitTesting.UnitTests.Tests {
	public class GlobalStatics {

		public static StringDictionary SiteDic = new StringDictionary(){
			{"allowDebug","true"},
			{"browserTitle",""},
			{"cacheHtml",""},
			{"cacheMedia",""},
			{"contentLanguage",""},
			{"contentStartItem",""},
			{"database","master"},
			{"defaultDevice",""},
			{"device",""},
			{"disableClientData",""},
			{"disableXmlControls",""},
			{"domain","extranet"},
			{"enableDebugger","true"},
			{"enableLinkedItems",""},
			{"enablePreview","true"},
			{"enableWebEdit","true"},
			{"enableWorkflow",""},
			{"enableAnalytics",""},
			{"filterItems",""},
			{"hostName","unitesting.local"},
			{"htmlCacheClearLatency",""},
			{"language","en"},
			{"loginPage",""},
			{"masterDatabase",""},
			{"mediaCachePath",""},
			{"mode","true"},
			{"name","UnitTesting"},
			{"physicalFolder","/sites/UnitTesting"},
			{"port",""},
			{"requireLogin",""},
			{"rootPath","/Sitecore/content"},
			{"scheme",""},
			{"startItem","/UnitTesting/Home"},
			{"targetHostName",""},
			{"virtualFolder","/"},
			{"xmlControlPage",""}
		};

		public static SiteContext UTSiteContext {
			get {
				SiteInfo si = new SiteInfo(SiteDic);
				return new SiteContext(si);
			}
		}

		public static string DefaultDeviceID = "{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}";

		public static Database MasterDB {
			get {
				return Sitecore.Configuration.Factory.GetDatabase("master");
			}
		}
		public static DeviceItem DefaultDevice {
			get {
				using (new SecurityDisabler()) {
					return MasterDB.Resources.Devices[DefaultDeviceID];
				}
			}
		}

		public static Language EnglishLanguage {
			get {
				return Language.Parse("en");
			}
		}

		public static void SetupContext(){
			Sitecore.Context.Site = UTSiteContext;
			Sitecore.Context.Language = EnglishLanguage;
			Sitecore.Context.Database = MasterDB;
			Sitecore.Context.Device = DefaultDevice;
			Sitecore.Context.IsUnitTesting = true;
		}
	}
}

and here's how I'm calling it:

[SetUp]
public void SetUp() {
	GlobalStatics.SetupContext();
}

The use of the .IsUnitTesting property is optional and will modify the path where Sitecore looks for the config files. Dan discusses this in more detail.

Subclasses

While working with web tests I found it useful to convert the TestSite object to a more specific subclass. This is because I added key/value properties to each of my Sitecore site entries to identify the site ID and content language. To be able to do this I created a SitecoreSite subclass from TestSite. Here's the definition for that:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using NUnitTesting.Core.Entities;

namespace NUnitTesting.WebTests.Entities {
	public class SitecoreSite : TestSite {

		public string LanguageCode {
			get {
				return GetProperty<string>("LanguageCode", string.Empty);
			}
		}
		public string SiteNodeID {
			get {
				return GetProperty<string>("SiteNodeID", string.Empty);
			}
		}

		private T GetProperty<T>(string key, T defaultValue) {
			return (Properties.ContainsKey(key)) ? (T)Properties[key] : defaultValue;
		}

		public string SCBaseURL(TestEnvironment env) {
			return (string.IsNullOrEmpty(LanguageCode))
				? BaseURL(env)
				: string.Format("{0}/{1}", BaseURL(env), LanguageCode);
		}
	}
}

The TestSite class has a ConvertTo method that basically just serialized the class the JSON and then back into the specified class. Here's an example of that from within a web test class:

SitecoreSite scs = ContextSite.ConvertTo<SitecoreSite>();

Automated Updating

After the system is setup and running the long term goal is to get the binaries/configs updated regularly. I haven't made it to this point yet, so I don't have anything to provide as a sample but it will require some sort of batch/build script to do it.

Fin

So that's TestStar. The web app / console app that can run both unit and web tests. Development is ongoing but at the current time of this writing it is stable so please download and fork away.

Prologue

This article is about the theory of template development. A topic, which at its best, is dry and wonkish. My goal is to provoke thought and discussion but mostly to prevent others from having to continuously wander through this hedge maze until they invariably come to the same end. Besides that, there may be a better solution still that someone else has come up with.

Act I

Recently, on a particularly frigid night in Boston, while milling around after the New England Sitecore User Group, I was speaking to Rick Cabral about his ORM Diamond. I mentioned that I thought he was right to conclude in using interfaces to describe datasource data types in sublayout controls. My reasoning was that fields should be grouped into templates by use, then inherited into pages. Each grouping would then lend itself to being defined by an interface which could be implemented on any page class and then many different page types could reuse or interchange code and presentation elements as needed.

Act II

Let me back up a bit and explain where I began when I came to my conclusion. I was thinking a bit about an article I read discussing the page template mistake which discusses better ways to build templates for the page editor. I really wanted to follow up with my own thoughts on page templates because there are a lot of constraints when building a multisite solution including template reuse.

When you're trying to share templates between two or more sites, you're going to hit page template dilemma. Let's say you have three sites in your system and two of those three will share the same presentation but will differ in content and language. The third site, however, will need to have a separate design and as a result will need a different set of presentation details. If you've already created a page template with fields and presentation set, you have two options: inherit from the page template that already exists and get with it all the standard values baggage (insert options, default tokens, and presentation details) or create an entirely new page template and duplicate all the fields. Both prospects are disagreeable.

Inheriting may leave you with fields you don't need and if the presentation of the base template gets updated, the changes may or not make it through since your customization will break the chain of inheritance. What you're left with is needing to reset the presentation details now and again to get the changes to pass through, then re-adding any changes you've made. Even now, I'm left with a foul taste in my mouth from when I had to do this.

On the other side of the coin, is creating a duplicate set of templates and fields which has the drawback of limits with respect to scale. One thing I hate doing is repeating myself. I have a particular distaste for duplicating templates and their fields since, with just over a hundred site in my system, the thought of a creating a hundred sets of templates, just so I can change the presentation, makes me ill.

Act III

This presentation problem has been with me for longer than I care to remember. Even back when it was only two or three sites in a system, there was a need to share template fields and separate the presentation by site. At first I thought about customizing Sitecore to support having multiple standard values and then setting up a custom Standard Values provider that could choose which to use depending on the site settings. Not entirely a bad solution but when you customize built-in template behavior you're asking to get your system wiped out when you upgrade.

I've also inherited a solution that modifies the path to the layouts and sublayouts during page load to look in the physicalFolder path of the context site and check for the file there before defaulting to a global folder. It's really brilliant but still creates a lot of customization to pipelines that a future change to Sitecore could break.

Denouement

To solve this problem in a way that's a bit more future-proof, you have to find a way to work with the system. The realization I then came to, was that you need to think of page templates as non-field, presentation only. This allows you to create more than one copy of the same generic page template but with different presentation settings and without duplicating the fields. The next logical step is to see each group of fields in their own templates that can be inherited by these page types. To clarify I'll show an example. Let's say we want just a home page, regular page and a form page across these three sites I've mentioned previously. Here's how I'd structure the templates:

To start I'm using a global pool of "Page Groups" that define fields for specific uses like meta data, page content and form handling. I've also created an intermediate layer with the "Page Base" and "Form Base" templates that then inherits these groups. Nothing up to this point has any type of presentation associated with it. They only relate fields to a use and groups of fields to a page type.

The presentation details are defined for each specific site on the matching "Page", "Home Page" and "Form Page" templates in those specific site folders. From the earlier example, the first two sites would use the "Default" set of page templates. The third would use the "Site 1" templates. Both "Form Page" templates will inherit from the "Form Base", "Home Page" and "Page" from "Page Base". Now there's no duplication of fields and you can specify the presentation for each site uniquely. From here you can create IFormBase and IPageBase interfaces that will allow you to build reusable web controls and methods in a core library and globally reusable sublayouts. And ultimately that is the goal; reduction, reuse and modularity.

Addendum

After reading through this a second time, I realized I forgot to mention how this affect insert options. This is a good place to add them because the root structure of what you'll be creating in sites, are pages. Or more explicitly, the items with the presentation set on them. You could brush some broad stroke with the insert options on the standard values here. Then in the event an individual site needed them, you could manage insert options on a site level as well. 

For those of you unfamiliar with it, the Data Importer is a Sitecore module on the Sitecore Marketplace. It's a tool that allows you to import data into Sitecore. It was originally built to import sites from existing websites by pulling from databases and into content items. It allows you the flexibility to handle each individual field in a row of data using a number of built-in conversion types but also allows you to extend the system by creating your own providers for data sources and adding your own field conversion types. 

I just wanted to write a quick post about the latest release. It's now on version 1.4. If/when you install the package make sure you do a merge on the system settings portion so your existing settings aren't wiped out. Also make sure you do an overwrite on the application content (core) or templates (master) because they need to be updated. If you're visual and prefer a video, I've posted one on YouTube explaining the latest changes. Otherwise, there three changes that comprise the update. 

First is that CSV file imports are now supported. There is a new template type and backing class to support this. To run an import on this type, you'd place the full file path in the query field of the import item and each "From What Field Name" field will be a zero-based index number indicating the column position. The files you're importing will have to live within the web application folder to prevent read permissions issues. 

The second change is a UI change. I've removed the application that manages the imports and instead moved that layout to a tab on the import item itself. This means you won't need to be logged into Desktop mode to use it and will hopefully make it a little more intuitive since you will be able to run the import with the item itself selected. 

The final change was from a user request for a feature to allow importing based on a schedule. To do this I've added a console application that can be called from scheduled tasks or elsewhere. I've also added a sample batch script under the \sitecore modules\Data Import\scripts folder to show how you would call the executable and fire off the import. It's a small step but I can see that being a very useful feature going forward. 

Antecedent

Friends, romans, countrymen... et al, the Setting Up a Sitecore Extranet article I wrote in 2011 is one of my most trafficked. It occurred that many of you could benefit from the additional changes that I've made since then, so that those who are setting up your own system by hand could now benefit from a module that manages most of the process for you. For visual reference, I've made a video for a quick intro to the module.

There already exists an extranet module on the Sitecore Marketplace but I'm not the developer of it. Sitecore is. The documentation says it was created in 2007 for Sitecore 5.3. I've never used it so I can't speak to its efficacy. I'm presenting my own solution and you're free to choose which will work best for you.  

The Gist

You can download the Extranet Module on the Sitecore Marketplace. You can also download the Sitecore install package from Github and the full source is available here.

There are two wizards in this tool that are used to setup and remove extranets from individual sites. This implies it's support for a multisite installation. Depending on whether you're using the Multiple Site Manager or not, it will create attributes for the site nodes in the content tree or in include config files following the pattern Sitecore.Extranet.sites.<SiteName>.config. These attributes will allow you to specify the security role for the extranet, the prefix that will be added to usernames (so the same user can have separate accounts on separate sites), the from address emails are sent from and the number of login attempts before being blocked. By default the setup wizard creates them individually for each site but you can change them in the config or content tree depending on where your sites are managed. I only recommend you do so only after you understand all of what's affected (during installation, the operation of it and removal process). 

When adding and removing sites, you may need to know which sites already have an extranet applied. To help you quickly determine this, there's a listing on the "Extranet Settings" tab of the system settings item under /sitecore/System/Modules/Extranet.

Pimp My Ride

There's a set of templates that represent the page types that manage logging in, registering and managing account information. The templates themselves do not have presentation or security setup on them, although the form sublayouts are provided. They are, however, setup on the "example" branch set of templates provided. I designed it this way because each site may require their own presentation and security settings. The security is only applied to the login page and has a generic role set on it which is replaced during creation by the wizard with the new role created for that site.

The intention is that you would create your own branch (you could duplicate the example) and configure your own presentation settings. The security settings should be copied since they are a token value that is replaced with the specific site role during creation. This will allow you have different configurations for different types of sites. When you use the setup wizard, you will select which branch you want to use for the site you're adding it to. One item to note is that there is an account navigation sublayout that you should apply it to every page on your site. Currently it was added to the login page presentation settings just so that you'd know it's there but the best way to display it is to add it to a global location on a page or master page file. The account navigation is where the logout and edit account buttons are located so it's needed for a fully cyclical system (login and logout). It will automatically hide itself when a user is logged out so it won't be obtrusive to the existing site.   

When working with the sublayouts, you have the ability to create your own front end. You can decide how best to handle this. You could just alter the files provided or create new sublayout items/files and attach them to a new branch template. The default files are stored under /sitecore modules/web/extranet/. 
Much of the functionality can be modified also. On the base classes for the sublayouts many methods are marked virtual so that, if needed, you can also override and customize that locally too.  

Throughout the sublayouts used by the extranet, there's all types of field labels, error messages and other text that can be translated. 

The settings section contains a dictionary of these text items. With this you're able to populate versions of the items into the language each site will be using. If this doesn't meet your needs or you already have a dictionary that you manage, you can override the form text provider in the include Sitecore.Extranet.config and manage the form text in your own way.

Wrap Up

Although this does work with Sitecore 7.1 it's still built using Sheer UI. I don't have a firm enough grip on SPEAK to have created a version using it so it'll be a while before that happens, although I'm sure I'll end up with a series about how to do it. 

Live long and prosper.

I've more recently released a video recap of the presentation I gave to the New England Sitecore User Group.

Homage

You've unlocked a door with a key of imagination. Beyond it is another dimension. A dimension of clouds. A dimension of sites. A dimension of mind. You're moving into a land of both shadow and substance, of things and ideas. You've just crossed over into, the Sitecore zone. (shocking music)

On a rather nice day in November, at a bar in Cambridge, after an event known as the Laidlaw Classic (annual birthday football game), an Oasis employee mentioned to me that he was taking a class on Genetic Algorithms. After questioning him to death, since I'd never heard of it, I looked up what he recommended and found a hello world genetic algorithm on GitHub. With it, I was able to take it apart and understand the mechanics. After finally realizing what it was doing, I was able to hotwire it, abstract it a bit, tweak it, and make it provide content for a website based on what type of links I'd been clicking.  

If this sounds like I'm headed in a particular direction, then you're right, it's personalization. I've written before about some aspects of personalization dealing with the rules engine to provide content but this could prove to be much easier. Instead of several steps including analyzing your analytics, a task list better managed by an algorithm, marketers and content editors would be free to focus on what they do best: creating killer content. In turn the technical tasks would fall to those who prefer to do them: developers.  

At the time of the release of the article, what I've built isn't ready to be deployed. It's a prototype. This is an open call to developers who are interested, to take some time and experiment with it. There's a few different uses for an algorithm like this that I can think of (Personalization, Device Detection, Caching and Security) but I'm sure there's more. Also this project isn't tied to Sitecore's personalization engine. It is free standing and the algorithm data and click events are only stored in session. It was a lot easier to build the protoype by limiting my scope and there are still some problems yet to be solved for this to be a package-able, turnkey solution. 

That being said, let's not be deterred. Let's shuffle on. 

Machine Learning

Ok, so at the core of this technology I'm using a Genetic Algorithm. This is branch of Machine Learning which is a branch of AI. It's not gonna get up and walk away on it's own but it can traverse a large field of data that you yourself couldn't. The field of data I'm referring to is the combination of all your users, their interactions and all the content available to them to view. Given a sufficiently large user base and content repository, you're looking at quite a large number of combinations. 

Manually creating personalization schemas will take you forever to get the level of detail the algorithm can produce. Sure, the algorithm isn't perfect but it will improve over time and since a user can change it's mind at any given time, what constitutes a "good" solution is a moving target. This is the real motivation for allowing an adaptive algorithm to do the work. Instead of constantly tweaking and tuning your personalization, the algorithm will continually adjust itself, leaving you free to spend the day doing more productive things.   

Genetic Algorithm

So what's going on behind the scenes? How does the algorithm work? Well, it's using a combination of random chance and probability. It's using random chance to create variability (randomized solutions). Then it's using probability to focus those variations into a productive direction. There's a number of percentage values that are used by the algorithm to set limits on things like mating, mutation and elitism. I've also added a few into my implementation for additional functionality. 

Populations

The algorithm starts by generating a population of entities known as Karyotypes. These entities will have a sequence of Genes. Genes can represent anything. In this case, they represent content categories; tags. The number of genes relates to the number of renderings you need filled. The Gene list of tags tells you what category of content to provide for the renderings. This is so that each rendering can have its datasource set with content from that specific content category. Given two renderings and three categories, an entity's Gene could look something like this: Category1-Category1 or Category3-Category2. The population will start with around 100 different entities and each entity will be a complete random mix of tags for genes. Over time, the population will partition a set of entities that best represent your clicking behavior. 

Fitness

The way the algorithm determines which entity to use as a solution, is to compare their fitness. Fitness can be anything you want but in this case, I've decided that the the total clicks for a category is it's value and that list of categories an entity contains can then be summed up. In my sample site (provided in the source code) I've used colors as tag categories. This was to make it easier to visualize what was happening. As the user clicks on navigation items, values are stored by category. The more clicks, the more valuable it is. The image below shows how a number of clicks for colors would be used to measure the fitness of entities for a page with 2 renderings. For example, since Red was clicked twice, it's value is 2 and a if an entity is Red-Red, then it's sum is 4. 

In this algorithm, I'm shrinking the value of a click over time so that more recent activity is more valued than further in the past. Big shout out to Andrew Blackmore at Velir for his post that I used as a model. By default it will not reach zero but a fraction of the original value. Only when you've clicked on something a sufficient number of times will the value remain above the threshold. In the image here, you can see the decay of two separate clicks on Red and how on the second click, the value begins to grow:

Genetic Algorithm Lifecycle

The fitness values are used to sort the entities and provide the best solution. The algorithm then starts to mate (mix genes) between entities and then re-sort. This is repeated until an "optimal" solution is found but in this case the "optimal" solution changes with the user's habits. 

Elitism

A percentage of the population at the "most fit" end is not replaced during the mating process so that the population will maintain its fitness or improve but never decline. This percentage is known as the elites.  

Mating

Mating will loop through proletariat (non-elites) in the population and randomly select a partner to mate with and replace itself with the child. The probability that a proletariat will end up mating is determined by the Crossover Ratio, which generally is greater than 50% since you want a good chance of mating. If an entity is chosen to mate, it randomly selects a partner and continues to try looking for a more fit parent a certain number of times. The number of times it will try is the Tournament Size. During this process it's possible the other parent could end up being an elite. When mating finally does occur, the process of mixing the genes between two parents, in a biological sense, is known as Meiosis.

Meiosis

To understand Meiosis you'll have to know a bit more about how the Genes are stored in an entity. This diagram shows the class hierarchy within my version of the algorithm. Each subsequent row expands the object into it's components: 

For an individual entity (Karyotype) there are two Haploids, one from each parent. They are made up of a Dictionary of Chromosomes. Each Chromosome is a List of Genes. Meiosis produces a new Haploid by randomly mixing the two in itself. This means that each parent will provide a completely new Haploid for a new entity. Here's an example of how a single Chromosome is combined:

Phenotype

When a new entity is created, it needs to determine what characteristics it will display. Only half of it's available Genes will be used. The other half remains hidden. The half that is shown is known as the Phenotype. It will compare the Genes from both parent by using the Dominance setting to determine the Phenotype. The Phenotype of an entity is what is used to judge its fitness. Each Gene has a Dominance setting, which is set randomly in population initiation. If the Gene's Dominance settings are the same, one is chosen randomly, but if they differ, the Dominant wins over the Recessive.  

Mutating

There's also a bit of random mutation thrown into the mix which provides yet more variability but too much can cause instability. It, like mating, is reserved for the proletariat section of the population. The Mutation Ratio is the probability that an entity will mutate.

Page Lifecycle

So now that you know how the algorithm works, how does it apply this to the page content? The crux of the situation is setting the datasource for a rendering to a value defined in the config and then during a pipeline processor, the datasource is switched out with a content item based on what the algorithm has chosen. Then of course there's a click event handler, which stores what tags have been clicked. Here's a high level breakdown of the chain of events. 

For a developer, you'll first want to create an algorithm definition for your site. This contains all the values needed for the algorithm to run and then add an attribute to your site node that tells it which algorithm definition to use. Each site can have it's own or they can share. From there, you'll just need to determine which renderings you'll want this be used on and set the datasource to a common value in the config. Above and beyond setting up the algorithm, you'll also need to have a tagging system. At least a folder full of tags and a field on both content items and navigation items. I don't recommend tagging any more navigation than you need to. Top level tagging is really enough. The concept of having the algorithm choose sub-category content would require a separate list of click event data and running the algorithm on it separately. I'm not sure how computationally expensive that will be but I imagine it could grow to be quite taxing. For now it's just a top-level category selector. 

For a content editor, their life is actually pretty cushy. All they need to do is create tags, tag content and then tag the navigation to line up with the content. The goal really was to make this a more user friendly experience.

The 411

Ok, so I've got the source code for my project, called SitecoreGeneticAlgorithm, on GitHub. It's broken up into several C# projects: GA.Nucleus (genetic algorithm), GA.SC (Sitecore specific adaptation), GA.SC.UI (Sitecore specific config, web service, layouts, and packages) and GA.SC.UI.SampleSite (generic site adaptation). The sample site should be replaced by your web app. It doesn't take a lot to wire up the system and it provides a working example of what you'll need to do. Here's a breakdown of the libraries starting at the top:

GA.SC.UI.SampleSite

The project really only contains a few files with little custom code that you'll need to change.

GA.SC.UI.SampleSite.config

The config is a standard site config definition with one additional attribute: gaName. This tells the system which algorithm definition to use. In this case, it's set to use the "gamanager", which is used in a diagnostic tool which I'll explain in the next library. You will likely want to create your own algorithm definition and tweak the values from there.

GASampleSite-releasedate.zip

The GASampleSite*.zip is the installation package for the site including the content sublayouts and templates.

GAMaster.aspx

The GAMaster.aspx is standard but with one addition: the click event handler. The way that user clicks are registered is through a web service. I'm connecting to it using jQuery. 

BaseSublayout

This class is used for easy access to the datasource.

GADrivenContent.aspx

The GADrivenContent.aspx is actually completely standard. The name implies it does more than just simply display the datasource but it doesn't. It's only responsible for displaying the datasource that gets set. Keeping this agnostic will require the search part of the Sitecore adaptation to be tweaked to match content types to rendering types. Currently it's expecting a known type. 

GAPage.aspx

The GAPage.aspx actually only displays the field content and doesn't have any algorithm specific function.

TagUtil.cs

The TagUtil.cs is used on the GADrivenContent page to retrieve the tag value from a datasource. It does this by taking the "Tag" field (search box field) from the datasource, converting it to items and retrieving their tag names. The "Tag" field is used to pair content with navigation elements so that when a navigation element is clicked a corresponding content item can be chosen. An example would be to have navigation elements like electronics, home goods and books. The electronics link would be tagged with "electronics" and then computers, phones etc. would also be tagged as "electronics" so that when the electronics section was clicked on, more computers, phones etc. would be displayed.

GA.SC.UI

This project also doesn't contain many files. I don't forsee a lot of changes to what's already there but there will likely be additions over time.

GA.config

The GA.config file contains all the information needed to run a Genetic Algorithm for Sitecore as well as class references for the algorithm, engagement values and a pipeline event that sets datasources on renderings. Here's a breakdown of the values:

• tagFolder - The ID of the folder (Bucket) where tags are stored
• contentFolder - The ID of the folder (Bucket) where content is stored
• contentTagField - The field on the content items that tags are set
• datasourceValue - The value of the datasource field that indicates it should be replaced
• chromosomeName - The name of the chromosome in the Haploid dictionary for an entity (only works with one so far)
• populationType - The class type of the population
• karyotypeType - The class type of the karyotype
• evProviderType - The class type of the EV Provider
• crossoverRatio - The probability the proletariat will mate
• elitismRatio - The percentage of population that is elite
• fitnessRatio - The percentage of the most fit value that is allowed to be chosen as a solution
• fitnessThreshold - The threshold the combined click events must be to initiate the algorithm
• mutatioRatio - The probability the proletariat will mutate
• tourneySize - The number of times a better parent is searched for
• popSize - The size of the population
• evModifier - Defines how click events decay over time
  • minRatio - The lower bound percentage a click value will reach
  • decimalPlaces - The number of decimal places a value should be calculated to
  • timespanAspect - The time frame the half life is based on (milliseconds, seconds, minutes, hours, days) 
  • halfLife - Half life in terms of Timespan Aspect. IE: 2 days. 

GAManager.aspx

The GAManager.aspx is a diagnostic tool to help understand the mechanics and test the algorithm without having to implement a site. 

GAModule-releasedate.zip

The GAModule*.zip installs the GAManager under /sitecore/system/Modules and the core content items, templates and sublayouts it requires. 

EventTracking.asmx

The EventTracking.asmx is the web service that is being called by the Sample Site to store click event data. 

GA.SC

This project does contain a good deal of the functionality and will be where most of the changes will be coming to. 

EngagementValues (EV)

There are three classes: DefaultEngagementValueProvider, DefaultValueModifier and EngagementValue. Then there are three interfaces these are based on so that you can provide your own. 

An EngagementValue stores a Value and a DateTime. The DateTime can be used to determine if a value is still relevant.

The DefaultValueModifier will take each EngagementValue and measure the value over time on a curve. The properties for this curve can be set in the config. The concept for this was pulled from Andrew Blackmore's post on Velir's blog. The idea behind it is to make a user's older clicking habits obsolete at some point in time. By gradually bringing the value down to zero, you're able to work against current activity. My particular implementation doesn't bring them to zero but instead a fraction of the original value. This allows the algorithm to still imply a value on something that has been clicked a lot in the past but not recently. Calculating values on a curve is an increasing taxing task and I currently am disabling it because it's too computationally intensive. It will likely be moved to a separate server to be run and queried.

The DefaultEngagementValueProvider is using a static (singleton) to store the values. This is hacky and will be replaced with a provider that supports a database for querying. 

Pipeline

There is an InsertRenderings pipeline processor called SetGADatasource, that fishes out all the renderings who need their source set by the genetic algorithm. It starts the algorithm and then tells it how many genes it needs (length of chromosome). Then provides it all the tags available to use as genes (IGene). The combination of the length and content for an entity is the Genotype. From a biological perspective, I believe I am using this correctly but it might be off. Feel free to correct me.

Web Service

The way that click events are tracked is by hitting a web service that stores them. This is implemented in using jQuery in the Sample Site. 

ConfigUtil

All of the config values are accessible through this class.

Algorithm Specific

The PageKaryotype, SCPopulation and TagGene are the implementation of the core algorithm library. The PageKaryotype is an entity in the SCPopulation. It defines a Fitness method using the TagGenes and EVs to sort the entities, which is the heart of how to guide the algorithm. 

The SCPopulation class manages storing/retrieving the population in session. This will eventually be changed to a provider system so it can be stored in session, file or database. Otherwise it doesn't actually modify the functionality in the DefaultPopulation class that it extends. 

The TagGene is the implementation of the IGene interface by providing the tag name as the GeneID. Genes are the building blocks of the life-imitating entity that is created. 

GA.Nucleus

This is, as the name implies, the core of the algorithm. The focus of this class is around Genes, Chromosomes, Haploids, Karyotypes and a Population. The biological jargon that I'm using is an attempt to be accurate to the model of nature, not solely as an attempt to be nerdful. But definitely somewhat because I'm a nerd... Aaaanyway.

Populations and Karyotypes

An entity in a Population is a Karyotype. The Karyotype is made up of two Haploids and a Gender(bool). The Gender prevents mating with oneself. It's two Haploids are from it's parents. Both the mother and father provide mixtures of their Haploids. 

Haploids

A Haploid is a dictionary of Chromosomes which adds depth to the detail an entity has. The sample site only uses one: PageContent. Although there could be multiple chromosomes for a page to represent different page areas (right and left column) or the types of content for a page (ad, form, informational). This is the base for institutional memory. The state of this information at any given time is the result of the time provided for it to traverse the information you've provided, the group of entities that constitute progress (elites) and the mixture of random possibilities (proletariat).

Chromosomes

The Chromosomes are a List of Genes. In the sample site the List of Genes are a list of tags that tell you what category of content to provide for that rendering. The content is currently done by simple Sitecore Query but will be replaced by an implementation of a search engine, like Lucene. 

Genes

Genes are only responsible for having an ID which is used to identify what it represents and a Dominance property which is used when combining the genes of the parents to know what it will express in the Phenotype.

Phenotype

The Phenotype is the set of Genes an entity expresses. It is used to judge it's fitness. It is a Haploid that is defined by comparing the parent's Genes and their Dominance. These are all set randomly in the population initiation. A combination of two Dominant Genes will choose one by 50/50 chance. Same if both are Recessive. But in a match between Dominant and Recessive, the Dominant will win out. The purpose for adding this into the system is to increase the amount of variation. An entity won't only have a set of Genes that it's displaying but will also contain an entirely hidden set. This information, although hidden, can resurface in a child upon Mating. It really allows a small population the ability to adapt. 

This stores the length and list of Genes to be used per Chromosome. The GenotypeList is a Dictionary of Genotypes that pairs with a Haploid. When the Haploid is created it uses the matching Genotype as a guide for what to construct.

Overall Usage

The Population, Haploid, Chromosome and Gene all provide a default class that can be used. The only thing you'll need to create, is an extension of the abstract class BaseKaryotype. I've provided this implementation in the GA.SC.PageKaryotype. This implements the fitness function and defines what makes a good result. These also are based on an interface so that you can replace them with your own implementations if desired. 

Eventually population size becomes a limiting factor. Data storage and algorithm processing will grow with a population. I have not yet found this limit and don't have recommendations on what is required to support any given number of users.

Another limiting factor is the Engagement Value Modifier. When calculating values on a curve, eventually the cost of measuring the asymptote can be demanding on a processor. There may be ways to cut it short through the calculation or simply by not calculating values past a certain date. It's likely this will need to be passed off and run as a batch on another machine to work at scale.

The end? Hardly. 

Although that's the end of this article, that's not where this ends. This is only the beginning of a long and interesting journey. I'm not going to stop until some robot from the future and crazy lady come busting into my home trying to make me stop. Your move universe. (squints at the sky)

References

In no particular order:

Viva Las Vegas!

I'm now watching mountains drift beneath the wing of my plane (sunglasses on). I'm leaving Las Vegas richer not poorer. You could say, I've just had quite an amazing Sitecore Experience (wink). 

The symposium held a palpable tension as the collective agreement was that we were participating in something much greater than what we could have expected. Each talk contributed to increasing awareness that Sitecore is no longer just visionary but enablers of everyone else's visions. 

Sitecore 8

Sitecore is now about experience management not just content management. They've expended a lot of resources engineering new ways to support the growing needs of business with insights and communication but with Sitecore 8, you now see the bigger picture. The UI has been redesigned entirely and what they've done with SPEAK will leave you speechless. For a taste of what I'm talking about, here's the new login page, application manager and content editor (pictures taken from slides and Sean Holmesby's rig. Thanks btw) :

The most immediate difference you'll notice is that there's only one way to login. After that it's using the app paradigm which you're likely very familiar with. Apps are now built using SPEAK so that even how you create apps has changed. And when you want to start building them, I was told several times and completely agree that Martina Welander has written great articles on SPEAK to get up and running. The most important thing to understand is that you work through Sitecore Rocks entirely. The structure is stored in Sitecore automatically for you and there's a lot of source binding functionality built in. Most of what you'll end up writing is javascript to manage the interaction. Oh and there is a pipeline to tap into... yes from javascript. Just let that sink in.

One of the details that isn't entirely obvious is that the underlying system is going to be getting a lot better in terms of modularity. The features that, in the past, have been most challenging to integrate should be getting a lot easier to tie into. There's a general rule that there should be generic connection points for all features so that they can be swapped out with competing technologies. It's more of a platform with an ecosystem of tools than an isolated ecosphere.  

Federation

I didn't catch Stephen Pope's presentation but it was talked about so much I feel like I did. Like the Borg, reaching out across space to assimilate all in it's path, you can use Sitecore to "Federate" your external (not in Sitecore) sites by applying analytics and personalization. As perplexing of a task as it may seem, the solution they developed, adding javascript to access the page, makes it seem absolutely easy. A testament to the amount of effort obviously invested.

Sean "Sherlock" Holmesby and Mike "_I_ See Interfaces" Reynolds gave one of, if not THE most memorable moment. I won't ruin the surprise for #symeu but needless to say I laughed so hard I almost fell off my chair. 

Jokes aside, most of the presentation touched on a host of powerful tools. From SIM to Powershell to RAZL and more, don't underestimate how much time you could be saving yourself. Knowing what tools are even available is just as important as knowing how to work with what you've got. Technology moves pretty fast, if you don't stop and look around once in a while, you could miss it.

Commerce Server 

The purchase of an eCommerce system was a smart move but making it use a generic connection so that other systems can replace it, is wise. I don't do any eCommerce so I spent most of my time attending other sessions but I can tell you what I learned tangentially; being able to integrate whatever platform you want shows that Sitecore is playing chess not checkers. I'm sure a good swathe of customers are selling products online and would love to be able to improve their customer experiences with personalization and derive insight from the analytic and path tracking data.  

Coveo Search

In a jaw dropping moment of clarity, Coveo is now offering a free edition of their search appliance. The concept is brilliant. For basic use, you're going to get an awesome search tool and when you want advanced features or support, you upgrade. I was skeptical at first but after seeing how easy they made the integration I'm sold. The feature set is extensive and I expect the number of people getting hooked on using it will be high. Here's a breakdown of the difference between the free and enterprise versions shown from their slides:

If you're like me and you're working with Lucene, here's a few reasons you may want to consider switching:

• no index lock issues
• simple as switching search providers
• sublayout components with configurable properties like page and result count
• results filter through sitecore security
• multi-lang support
• developer support available
• suggested results
• facets

Project Skynet

Jim Ward a.k.a. @jerrong a.k.a. Miles Dyson a.k.a. bucket boy (deal with it) a.k.a. the wiz a.k.a. Tim gave the talk that I was most intrigued with. Since my summer was spent thinking about how to integrate machine learning into Sitecore, what I saw affirmed that it was time well spent. True to form he presented deftly and with wit. For sure I'm several steps behind him at the moment. There's the concept of training models (supervised learning), integration with goals and patterns in xDB (db formerly known as DMS, OMS etc) and a different train of thought along how I was using machine learning. 

Until I get my hands on the Sitecore 8 release I can only explain what I saw but my interpretation is to have users create content and set the algorithm to a/b test it. Over time the algorithm can provide insight into what content could be used in other areas to improve engagement. My gut reaction is that it should go one step further and just let the algorithm automatically test all new content. Allow it to take the individual analytics for a user and match/compare with successful group statistics to make decisions. Ignore what doesn't work and use what does then provide a report to the user. People can then pick out patterns in the reports and modify their creation strategies accordingly. But again, I could very well be saying what they're already doing. 

Final Thought

Sitecore has begun turning itself into a hub of technological business solutions; all hot-swappable. The new UI will lead to ubiquitous device support and expect custom application development times to drop. Designing an amazing experience is now the new normal.  

I'm quite literally, so furiously, thinking of ways to explain the funky stuff inside my brain. I couldn't explain! You might understand though. In fact, I'd wager you will. 

So I manage a modest multi-site Sitecore solution. There's a lot of shared code. There's also a lot more overrides. At a certain size, a system must be tested so that I know I'm in control and not my old nemesis, EEENTROOPYYYYY !!! ... ; * ' , . (ahem) The only way to know for sure if some piece of code is working properly, is to test it. 

Test coverage, of course, comes with costs. Time to design it, time to build, time to test it. I've been building web tests for this system for years now. I rolled them into a project last april on GitHub named TestStar that uses NUnit and included some unit testing. It's a web app that works beside my Team City server. After a build, unit tests were run. Then the integration environment was updated and web tests were run against it. This offloaded the work my desktop was doing to a single test system that I could then also run in regular intervals. Like daily. Fed to an RSS feed. So I had a history. The tool works well and does an awful lot of work for me. But I now have a second case for its use, and eventually, hopefully I'll find a third. 

Over time I've accumulated a few modules on the Sitecore Marketplace. Being able to successfully manage all of them is beginning to require I automate my testing. Mostly in this case in the form of unit testing. I typically run a few version (numbers) of Sitecore. I include my projects from a gitroot folder. This way I can include each, multiple times into other solutions. If I add tests to the projects, when I run a build, all needed files for testing can be in any given environment. The easiest way to test this is to have a modular Sitecore <site> that tests itself and other sites. In this way, it's consolidated PLUS (big plus), it can query for items and quite thoroughly test live content. My first iteration of TestStar also had batch scripting ability, if I wire that up in some friendly way, and or Powershell scripting, I'll be back to sleeping at night... (maybe). 

After a time, I plan to release the latest source as a module, fully tested and ready for the daily grind. Part of this involves doing some of my favorite work: UI, UX or whatever nomenclature you prefer. For this I'm riffing off Mass Effect 2 and DJ Jazzy Jeff. Haven't decided which background works better. Here's where I'm at:

Thoughts?

Why: adverb; for what reason or purpose. 

It may come as a surprise to some of you that I'm not an Automata. Surprise! I'm a human being. Unlike a robot or even a cyborg, I can't instantaneously learn Kung-Fu by downloading it. It takes time to accumulate knowledge and experience. I have to read, try and sometimes fail before I learn about something but eventually with persistence, I get there. 

For those readers who are not human, this process of learning begins with why. The pursuit of why something happens is what leads to understanding, not the other way around. I often find the why conspicuously absent from technical articles, which focus solely on the how. Not that there's anything wrong with that, mind you. Sometimes that's really all that's needed but other times, what usually starts as a journey, can end up becoming a mission. Searching, scrounging and sifting until, hopefully, I understand why.

Dependency Injection (DI) and Inversion of Control (IOC)

I'll admit, I had no practical understanding of the terms. I heard them mostly through a bluster of buzzword salads which I find unappetizing. (I'm wordy but it's a means, not an end) This time though it came up recently several times so I curiously began querying for answers. Sure enough, High Priests are already condemning it's lack of ubiquity so I felt as though I MUST be missing something big. I was sure it was some sophisticated strategy, tantalizing technology or epistemological epiphany. Why else would it be held in such reverence, such obfuscation, such... prolixity?  

Dependency Injection(Wikipedia): is a software design pattern that implements inversion of control for software libraries, where the caller delegates to an external framework the control flow of discovering and importing a service or software module.

Wow. Yeah. Ok. Sure. Wait, what? Reading this is taxing. Like I mentioned previously, I'm not a machine so I have no interface for language this obtuse. It's like being asked for the time and responding: "The 4th dimension is exactly 33/60 through 18/24 of a rotation of the earth on it's axis." Accurate only technically but borderline gibberish. Certainly of no help to anyone looking to understand this seemingly inexplicable topic. 

After finally finding a thoughtful explanation, (most were so vapid as to leave me more confused) I was at first a little doubtful what I read was the answer. Why was there so much ado about using interfaces? It's one of the core components of all object oriented languages. Surely there isn't a design pattern SIMPLY TO TELL YOU TO USE AN INTERFACE! But sure enough there is. It's worth mentioning that with the paucity of boundaries given in most articles, one would be left thinking that you should be replacing every class reference with an interface. EVERYWHERE! Just go out and double the size of every library with interfaces. No limit. Why? Because, you imbecile! Dependency Injection! (face palm) 

Absolutism is a poor, poor practice which leads to poor outcomes. I certainly don't know everything but I do know one thing: when to use an interface. Perhaps it's time to explain a bit about... what's the word.... damn... uhh.... oh right; WHY! So here's my take:

If you use interface types as method or constructor parameters, those parameters can be swapped out with any implementing object available in any given situation. If you ever even knew why interfaces existed, you probably already understand the basics of this concept but if you don't, no worries. Living is learning. Let's forge on with an illustration using a good 'ol car analogy. If you were to design a car with wheels permanently fixed to the car, you'd be buying a new car every time you get a flat tire. However, if you design a car to work with a standard wheel interface (four or five lug nuts), then you'll be able to swap them out with snow tires, rain tires, 50" rims or whatever your heart desires. The point isn't to "have a caller delegate to an external...." or other vague rhetoric intending to sound smart (sigh). The POINT... is to make a modular framework that is flexible and reusable.  

Interfaces describe the minimum requirements to be functional. If your class provides this functionality (known as implementing the interface), it'll fit right in. But again. If you were only shown how to use an interface, you might miss the forest for the trees. Knowing why will help you understand that you don't need to replace every class reference with an interface. You'd infer that you only need to consider replaceable parts. Again with the car analogy: If you damage the frame of a car, you're not going to find after-market parts for a frame. It's simply not a replaceable part. Making it replaceable makes NO sense. But again, not everything is a car. Heck I've even written a class library almost entirely consisting of interfaces! So use your best judgement. 

Test Driven Development (TDD)

So if you ask around, it's all the rage to be practicing Test Driven Development^TM. Everything is better with Test Driven Development^TM. It's so easy to start Test Driven Development^TM. Test Driven Development^TM saves lives. Test Driven Development^TM cures blindness. Test Driven Development^TM can even make you popular! You're not already using Test Driven Development^TM? What's wrong with you?!?! Oh right, repeating something tirelessly, breathlessly doesn't help you understand why you should do something. 

So consider if you will, the lifecycle of designing software. Testing what you've built should at least cross your mind. Once you've been burned badly enough by maintaining untested or manually tested code, you'll probably want to seriously consider automating your testing. Then once you begin automating your testing, you'll find it easier to incorporate testing into the design from the beginning. Voila! Test Driven Development is born. It's more than just an obnoxious phrase used to bludgeon any and all conversations into a stalemate. It's actually helpful.

So now that we know why, what does this have to do with DI and IOC? Well, to effectively manage testing any code, you'll need to isolate the core of your code from the implementation. In a web application, the core is generally found in the libraries. The implementation is generally in the sublayouts, web services and other end points. In a unit test however, the end points are irrelevant. You can, and I do, test end points with web tests, but I really want to focus on the unit testing here. To be able to run your code outside of a live environment means you'll have to replace situational content from databases and web services with static, testable components. To do so, you'll need to make these inputs provided as... wait for it... interfaces.

Wow. Full circle right there. Development begets testing. Testing begets interfaces. Interfaces begets modularity and apparently it's very own design pattern (shrug).

Okay, so one last thing about testing because, I've been hearing conflicting opinions on what you should focus on when testing. In my, albeit limited, experience, I've come to align myself with the mindset that tests can serve both as checks on your system and as documentation. In this way of thinking, if it in anyway shows the next guy(or gal) how to instantiate and use your code, it's helpful. Most importantly though, testing is about stability. If it helps you maintain a stable code base, do it. I'd rather have too many tests than not enough.  

A Few Minutes with Andy Rooney Mark Stiles

Alright, I'm almost off my soapbox. Here's where if you didn't get the sarcasm and insinuations, I spell it out for you. If we never learn to question why we do things, it's no longer a science, it's a religion. There's no nuance, only "The one true way". What's the answer to the unique problem you're faced with? Apply approved patterns. Don't think, just do. To be sure, there are plenty of optimal approaches for certain types of problems but I'd rather know why things work or don't so that in the event my situation is unique, which it ALWAYS is, I know how to adapt what I've learned. I'm not a compilation of things I've memorized. I'm a human being, not a robot doing. So don't be afraid or discouraged to try something new or fail. Encourage exploration, collaboration and most of all, discussion.

</RANT>

Not a very long time ago,
in a repository not particularly far away....

It is a period of automation and obsolescence.
Sitecore developers committing
code, day and night, globally.
Spiraling like the arms of a galaxy
sharing tasks, techniques and trade-craft.

During the past few months
I've been developing my 
ultimate tool, the TestStar
A sitecore module with enough
power to test an entire platform.

In pursuit of stability whose breadth
and depth I've not yet known.
To restore freedom for productivity to all.

Far Out On The Outer Rim of the Galaxy...

I've reached the end of a long road. I've been building and rebuilding a testing tool for myself for roughly four years now. It began as a web testing Visual Studio project, then morphed into a stand-alone web application with batch script support and unit testing running against NUnit. It has now has grown into a Sitecore module that installs as a web site and can be run from Powershell scripts.

The battle station is heavily shielded and carries a firepower greater than half the starfleet. In it's core reactor system is a set of manager classes that you can request to run tests. I do this through a web service call from javascript. It allows me to run them single-file and post the results as they're completed, giving quick user feedback in the event (especially while web testing) that your system is throwing errors. It also allows me to call the web service from a Powershell script. In this way, tests are running in a Sitecore instance with access to stored information that you can test with, such as IP addresses, language codes and redirect domains.  

The draw for me is that it provides a framework for unit and web testing within a Sitecore context that can help nurture test driven development. This force can have a strong influence on the weak minded and if you were planning on having your junior developers do the testing by hand, those are not the droids you're looking for. A side benefit here too is that it can accelerate development by providing stability controls and automation handlers. I can now do the Kessel run in less than 12 parsecs. I do point-5. That's light speed.  

Here's a diagram showing how the application flows:

1. User engages with test forms
2. Form contacts web service
3. Web tests are run
4. Unit tests are run
5. PowerShell scripts are generated
6. Scheduled Tasks run generated PowerShell scripts
7. PowerShell contacts web service to run tests
8. Test results can be read from RSS feed

The Dark Side

You might be skeptical about ANOTHER module that you have to keep up with AGAIN. Well, consider this, if your empire doesn't have a tight defense, bugs will create weakness in your web application. If your approach is manual testing, it will not be easy. The target areas can sometimes be only two meters wide. Which requires a precise hit. Only a precise hit will setup a chain reaction. I know what you're thinking. "That's impossible even for a computer." But it's not impossible. I used to bulls-eye womprats in my t-16 back home. They're not bigger than 2 meters. 

... Wait... Yeah. I might be getting a bit off course. What I meant is that although, it seems like just another thing to keep track of, it's additional value is well worth the effort. 

Sure, there's all sorts of ways to run unit tests but having multiple tools allow you more flexibility and a greater variety of testing surfaces like through web testing. This tool can also run groups of tests so you can include unit tests provided with installed modules. I'm not sure how widespread that practice is yet but I'm including them within this module and have plans to do it for others I manage. If you want to be able to call these tests as a single effort it might be easier to keep up with a powershell script that can be auto-generated. I'm just saying.     

Engage Your Tractor Beams

If I'm beginning to pique your interest, there's a few ways to get started:

• Check out a demonstration of the full power of this battle station.
• Download it from the Sitecore Marketplace.
• Join the Developer Alliance with the TestStar source code on GitHub.

With the TestStar, no web system will dare oppose you now. You may fire when ready and may the force be with you. 

I always knew Sitecore provided Dictionary support for a long time but I also knew there were, shall we say, scalability problems. Like, let's say, general user access (it was located under the system node) and the inability to create individual dictionaries for individual sites. Because of that, I, and apparently others, have been inanely custom rolling dictionaries. That is, until I came upon Scott Mulligan's article highlighting the latest changes in Sitecore 6.6 (yesss, it's been a while, I know [sighs, rolls eyes]) that convinced me to finally stop reinventing the wheel and adopt the Sitecore provided solution. 

There are resources out there to help get started like John Wests' seemingly canonical post. There's also a super handy article by "sitecorecode" (I couldn't find the author's actual name but I guess you know who you are). And my long time friend M3, as he's known colloquially, breaks it down like a legend.

That being said, there's no cookbook so any real depth on the topic is up to us/me/we/you/them/it to decompile and sift through the debris [duh-briss], then craft clearer guides from zero to working. 

1. Setting up a dictionary

First thing you'll need to do, assuming you have a Sitecore instance running and you're logged into it (because without that you're basically reading the wrong site), is insert a "Dictionary Domain" [dik-shun-air-ee doh-main] object from the template: "/sitecore/templates/System/Dictionary/Dictionary Domain". There are no icons for the dictionary items so you'll just have to deal. However, since the larger structure is unlikely to change without a developer, you could probably get by with changing the icons on the instance and be ok. In case you're wondering, the red book is in the "People" [peep-pull] icon section. Second row, all the way to the right. It would however be a lot better if someone at Sitecore (wink, nudge, cough......) were to add the icons to the system itself. Fabulous even.

Dictionary Domains

Moving right along. Once you've got a "Dictionary Domain" (not scare quotes) created, you'll need to specify which dictionary your site is going to use. The way to do this is to add an attribute to the site node in your config file: dictionaryDomain="NameOfDictionaryDomainItem". 

If you're wondering how this helps Sitecore find the dictionary in the tree, I'm assuming (perhaps incorrectly) that the underlying code is using a "Lucene" [lou-seen] index to get all the dictionary domain items and pull the correct one by name. Probably.

Fallbacks

There's a fallback concept with dictionaries that should be explained. On the Dictionary Domain node you created is a "Fallback Domain" [fall-bak doh-main] field. If you were to have local (to the site) dictionary and a global (to mulitple sites) dictionary, then this would make sense to have a fallback. Maybe you want to have most of your content global but then a few overrides locally. The field seemingly also pulls results from Lucene allowing you to place your dictionaries wherever you need them.

2. Adding entries

Once you've got the basic structure in place, you'll want to populate some values. You have two insert options: a folder and an entry. 

Foldering

I like to separate into groups by function such as form labels, global labels, navigation etc. I'm going to create a single folder called global and a single entry called copyright. This is pretty common for a site so that should be easy to follow.

The Keys to the Car

When you create your entries, the name should be short and common sense. It's only going to be used by content editors to logically find content. On each item you have two values to populate: the key and the phrase. 

The key is what will be used to call this entry from code. It's a shared field so you won't have to set it on each language you translate to, thankfully. For the key value, I think it makes sense to use the format folder.subfolder.entryname. If we included the site name we'd hobble ourselves when it looks for fallback values. So in this instance it would be "global.copyright" [glow-bull dot kop-ee-rite] (since I didn't create any level of subfolders). I also recommend using lowercase since that introduces less room for error.

The phrase is the text value that shows up. This is not a shared field so it's value is unique to all languages used. Translate away!

3. Accessing the entries

Now the last step is how to access it where it's needed. You're going to be using the Sitecore.Globalization.Translate namespace. There's a list of methods available but mostly you're only going to care about a few. And they are: 

Sitecore.Globalization.Translate.Text(

string key,
params object[] parameters

)

This is the most basic query and it will look at your context site and try to find the "dictionaryDomain" attribute to find the set of keys to use. If you didn't set it on your site node it will default to the global dictionary (in Sitecore) and if it doesn't find the key there (which is likely), it will return you the key itself. If you think that's a bad user experience and don't want the keys to show up as a default, Mikey "Interfaces" [he-see-them] has a nice write-up on how to fix that. 

There's one thing you should wonder about and that is what do the object[] parameters do? Good question, I'm glad you were paying so close attention. This is a hidden gem that allows you to pass in parameters for the entrie's phrase field which is fed through the string formatter. In the previous step, for example, I'm passing in the year and having it replace {0}. Here's the honey:

Sitecore.Globalization.Translate.TextByDomain(

string domain, 
Sitecore.Globalization.TranslateOptions options, 
string key, params object[] parameters

)

This method assumes you haven't added any attribute to your site, or want to reach into a different dictionary than the context is set to. You specify the "domain" which is the Dictionary Domain instance item name. 

The TranslateOptions allows you to specify the Database, FallbackDomains, SuppressDefaultFallback and SuppressFallback. I can see this being useful for several types of edge cases. Sitecore modules come to mind since you might need your content to be in the core database or you might want to prevent fallback to dictionaries outside of your control. 

Also supports parameter goodness.

Sitecore.Globalization.Translate.TextByLanguage(

string key, 
Sitecore.Globalization.TranslateOptions options, 
Sitecore.Globalization.Language language,
string key, params object[] parameters

)

This builds further by allowing you to specify a language to get the content by. I would imagine that if you have a translated site but always wanted to keep, perhaps, your business name pulled from one language, you could. 

In Summation

The process is a little jarring at first and without icons, can be slightly less visually appealing but it's now (as in a while ago) possible to create a dictionary, specific to a site, populate unique keys that are translatable and control how to display them. Better late than never amirite? I meant me using it, not Sitecore changing it. Well, I guess that too, but also me. Anyways [that's-all-folks].

When you enter a battlefield, be it land, mind or disk, be aware (wink):

"No plan survives contact with the enemy"
-Helmuth von Moltke the Elder

That's not to say you should have no plan though. Quite the contrary:

"Victorious warriors win first and then go to war, while defeated warriors go to war first and then seek to win."
-Sun Tzu

The War of the Folders

Starting at the top, you've got to upgrade the updater so you can install updates. You'll be copying in new files. You've got to run SQL scripts. Then you fix everything that broke. If you cross more than one point version, well then you're doing this all, multiple times.

If you've done yourself any favors, you've downloaded the packages, scripts, updates, a "Site Root" zip of the version you're installing and last but not least, the steps for each upgrade path and broken them up into folders like "7.0 to 7.1". You should also consider removing Sitecore from your solution (except configs and binaries) and publish onto a stock installation. In my experience, you're best bet is to upgrade to the nearest point version and jump from initial release to initial release. 

Hopefully your config modifications from a stock build will be in a separate "include" config far away from the ground-zero. In a quiet, zen-like state. This state is within you. It is you. Well, at least, if you want to have any hope of survival. But really, this is because the heart of the battle lies in preventing a cataclysmic collision of config files. In your hands are your helmet, sword and shield: Winmerge, Visual Studio and Git.... Silence. My pupils dilate and the caffeine is flowing.  

"He who knows when he can fight and when he cannot, will be victorious."
-Sun Tzu

At first contact, you're looking each other square in the eye. There's no illusions about what you've gotten yourself into. You've entered the Thunderdome (shout out to all you Aussies). It's looking at you. You're looking at it. Neither speaks. Neither blinks. You breath deep and dive in. You get a quick thrill knocking down the first few dozen line ending changes. Alt-Down. Alt-Left. Alt-Down. Alt-Left. Oops, CTRL-Z. But after a time, you're just playing a two-bit game of Chuck Norris, Roundhouse Config Kicker. WHACK! WHACK! WHACK! WHACK! You become numb. The tension is worn away in an anti-climactic grind.  

Suddenly something pops up. You find the new default values have changed. Oooh. Your mind is mildly sated. A small meal though. No more.

"He who is prudent and lies in wait for an enemy who is not, will be victorious."
-Sun Tzu

As time wears at you like a sweaty shirt, you ease new entries into place. You delicately tip-toe through custom config sections. Bob and weave around modules and rewrites. Man-handle handlers and services. You win some, you lose some but you live to fight another day.

Apocalypse How

In the process of upgrading a system through 7.5 there's a slight bump in the road: xDB. Or rather, MongoDB. The xDB overview doc is a good start to understanding what's going on but long story short, you need to download, install, configure, run as a service, setup connection strings to it, modify the analytics settings, setup at least one goal so that it begins tracking (I couldn't find an article anywhere explaining this) and last but not least deploy it. 

Like a good soldier, Dan Solovay has written about Sitecore moving to xDB and a great piece on MongoDB with Sitecore and Nick Wesselman has a whole series on it. 

The highlights are the four "Analytics" databases (analytics, tracking.live, tracking.history and tracking.contact) that run on MongoDB and one "Reporting" MSSQL database whose file is named "Analytics". It's the one that comes with "Core", "Master" and "Web". That's confusing, you might say, why isn't it named "Reporting"? Because it's your old analytics database that's been converted. Well, I mean that you're going to convert/rebuild it right now (which is what the reporting.secondary connection string is for.). If you need help, tactical options are available. 

Ok, let's assume you figured all this out and your local environment works, what's going to happen when you go live? You have to deploy MongoDB somewhere secure, that your other environments can access. Maybe there's a CLOUD solution for all that BIG DATA, but I have to roll my own so I wouldn't know. If you decide to manage it yourself, MongoDB has a good deal of documentation but it's might require some translation (they've been changing their API with newer versions too) so you'll be spending a good deal of time googling, guessing and testing.  

If you make it all the way to Sitecore 8, which is what I'm doing, you should also be aware of a new tool being added called Phantom.js. You should download this from Sitecore's Dev site and not the product's site. It's contents are added to the /data/tools folder which is where some core pipeline expects it to be. This is used to take screenshots when the system is testing, or as I found out, it runs when I update presentation details on standard values.  

Enemy at the Gates

If you made it this far, "Achievement Unlocked: Dragonborn". But don't get too comfortble. You're out of the quagmire but your still in the trenches. The light at the end of the dark tunnel though, is an end-boss. One with more comebacks than King Koopa: IIS. 

You dive into your foxhole, reach for Visual Studio, load your solution, cock back NCrunch and start launching builds, like a battery of artillery fire. 

Publish-Development..... The output window gives you a mime's version of atomic gears grinding. The silence is deafening.

MISFIRE: Sitecore namespace cannot be found. Awwwrrrr. (update the reference to new binary). 

Publish-Development..... 

MISFIRE: awfffwwah (projects target the wrong .NET version) 

Publish-Development.....

MISFIRE: Could not find add method: SetCommitPolicy. Should I call in air support? (squints. raises eyebrow. scratches head. glances at Stack Overflow) 

Publish-Development.....(sound of a missile launching)

SUCCESS! A DIRECT HIT! 

You're palms are sweaty, you alt-tab to the nearest browser (never to IE though, just out of spite). CTRL-F5. 

"Config Error Cannot read configuration file because it exceeds the maximum file size"

Pfffwa. Buh. Muh... Guh...

REALLY?!?! Not a mis-spelling. Not a double-entry. NOT EVEN A SINGLE MISSING BINARY?!?! Wow. You NEVER, and I mean NEVER see that one coming. 

"Thus, what is of supreme importance in war is to attack the enemy's strategy."
-Sun Tzu

So now you're on an epic side-quest to peel off as much of the config into includes as possible. Oh don't forget, if you use config transformation (.NET not Sitecore), the build can't follow Sitecore's config references to the "include" config files. They haven't been joined in holy matrimony yet so be aware of that minefield. If you need some suggestions on what to move, I think it makes sense to pick sections that don't differ by environment but are fairly large. It was enough for me to move <databases>, <events>, <mediaLibrary> and <log4net>. For guidance just follow how the commands.config is managed.

Back to the future, you're heart is beating intensely. You straighten up in your seat. CTRL-F5. We've got to get it on the run! Grab the exception and find the nearest google. Furiously read the saveragery of battles past and learn from the bloated body of blogs and Stack Overflow. One, Two, Ten, Infinity. Found it. What was .... awwwwrrrrr...my fault. 

"The supreme art of war is to subdue the enemy without fighting."
-Sun Tzu

All Quiet on the Testing Front

Like a fever breaking, the page loads. You can't hardly believe you've done it. It's only one battle in many for victory but at least it wasn't Pyrrhic. Now that's not to speak of all the features that aren't quite working correctly like the "Experience Editor" (How do you choose which site you want to edit from the home page?). Or the Lucene indexes (I was using the scSearchContrib which is now deprecated). And let's not forget, caching (I had it customized).

Reconciliation

The update process is a work in progress but know that as technology advances:

"The wars of the future will not be fought on the battlefield or at sea. They will be fought in space, or possibly on top of a very tall mountain. In either case, most of the actual fighting will be done by small robots. And as you go forth today remember always your duty is clear: To build and maintain those robots."
-The Commandant, head of Rommelwood Military School 

In work, as in war, although the best laid plans of mice and men often go awry, this is your upgrade guide, so you won't be forced to hire Seven Samurai. 

"Make an upgrade plan if you're soon to, and defeat your chances of ending with Seppuku."
-Mark Stiles

It's been a while since I've updated the Caching Manager, which is a module on the Sitecore Marketplace, so during a recent upgrade it was about that time. I'm using the design from Sitecore 8. It's backwards compatible through Sitecore 6 (SIM makes this possible) so there's no excuse for not getting it. Unless, you're on 5? Was caching on 5? I honestly don't remember or probably never knew.

The what now?

If you've never seen the Caching Manager, it's a single .aspx file that can be used to view your cache settings from a running Sitecore instance. Sitecore provides a default at: /sitecore/admin/cache.aspx... buuut it's kinda meh. Well, really meh. When I started using caching, I couldn't clear small chunks out when I needed to, so I built what I needed. Just for comparison, here's the default:

The Caching Manager is a bit different. It gives a few general sizes but also a search form and some clear options:

• Search All Cache
• Clear Search Results (whaaaaaaa)
• Clear All Cache

Are there more Features?

Well, right, yes,

• Sitecore 8 design aesthetic (ooh doesn't it feel new)
• Sitecore protected page (requires login so it's good on prod)
• Separate tabs for different types of cache (more focused search)
• Search for either summaries of a cache or it's individual entries
• Select all buttons! (it's such a help)

My favorite tab "Caches By Site" lists the site caches by type: filtered items, html, viewstate, xsl. System sites are separated from custom sites so you can select those quickly too (kinda grainy, sorry).

"Caches by Database" similarly allows you to search by type: blogIDs, data, items, itempaths, paths, standardValues. This image shows individual entries.

I believe "Access Result Cache" stores obfuscated user data so you won't get much from it. I added it to be consistent and for what limited control you can have.

I separated Data Provider Caches just because there were so many. I never use it, but you might!

Miscellaneous is just that. 

Have you? It's okay if you haven't.

If you haven't used HTML caching (I have no clue how many do, but I'm curious to know) you should at least know how to start using it (aside from using the Caching Manager, which I highly recommend).

Select a standard values item with presentation details applied and select the top nav "Presentation Tab". Then click on "Details" (closer to the left).

In the list of the sublayout controls, click on one. ex: BasicPageContent

You'll see the properties for that control. If you collapse the "General" tab or scroll down, you'll see the "Caching" section. Clicking "Cacheable" makes it cacheable (of course you'll have to publish these settings for web). There's also a sub-list of "Vary by" settings and one event trigger "Clear on Index Update" which seems pretty self explanatory. 

The "Vary by" settings controls how many variations of the page are stored in cache. This page type, for example, will render HTML differently depending on the instance item. In this case it makes sense to "Vary by Data" so that all pages don't display the first page of this template type entered into cache. You can optionally store versions specific to the device, user, etc. 

HTML Cache

While we're on the subject of caching, let's talk about HTML cache, and clearing said cache (invalidating for pedants). In my day-to-day I'm mostly concerned with how the HTML cache performs. I prefer to have as little "thrashing" in the cache as possible; meaning if I publish one item, then I won't need to clear the entire HTML cache. This is how the default behavior works. In fact, the default behavior expects you to add entries to the config for each site you want cleared on publish events. I don't see the need in all that duplication. The sites already exist elsewhere and I want to cache them all. 

To rectify my needs with the will of Sitecore, I patch over a few cache clearing events to wire up my own class. The class will attempt to find the site that contains the root item being published by comparing content start paths with the item path. I'd considered trying to line up media library items to the site too but each system could be different in that respect. Consider it for yourself though. 

<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
	<sitecore>
		<events timingLevel="custom">
			<event name="publish:end">
				<handler type="Sitecore.Publishing.HtmlCacheClearer, Sitecore.Kernel" method="ClearCache">
					<patch:attribute name="type">Sample.Publishing.HtmlCacheClearer, Sample</patch:attribute>
				</handler>
			</event>
			<event name="publish:end:remote">
				<handler type="Sitecore.Publishing.HtmlCacheClearer, Sitecore.Kernel" method="ClearCache">
					<patch:attribute name="type">Sample.Publishing.HtmlCacheClearer, Sample</patch:attribute>
				</handler>
			</event>
		</events>
	</sitecore>
</configuration>

Here's the class they reference:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Collections;
using Sitecore.Diagnostics;
using Sitecore.Sites;
using Sitecore.Configuration;
using Sitecore.Caching;
using Sitecore.Events;
using Sitecore.Publishing;
using Sitecore.Data.Events;
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Web;

namespace Sample.Publishing {
    public class HtmlCacheClearer {
        // Properties
        private readonly ArrayList _sites = new ArrayList();
        public ArrayList Sites {
            get {
                return this._sites;
            }
        }

        public HtmlCacheClearer()
            : base() {
            _sites = new ArrayList();
        }

        // Methods
        public void ClearCache(object sender, EventArgs args) {
            Assert.ArgumentNotNull(sender, "sender");
            Assert.ArgumentNotNull((object)args, "args");

            Item rootItem = null;
            string argsType = args.GetType().ToString();

            if (argsType.Equals("Sitecore.Data.Events.PublishEndRemoteEventArgs")) { //for remote endpoints
                PublishEndRemoteEventArgs pargs = (PublishEndRemoteEventArgs)args;
                ID did = new ID(pargs.RootItemId);
                Assert.IsNotNull(did, "publish root item id");
                Database db = Sitecore.Configuration.Factory.GetDatabase(pargs.TargetDatabaseName);
                if (db == null) {
                    Log.Info("HtmlCacheClearer: " + pargs.TargetDatabaseName + " db is null", this);
                    return;
                }

                rootItem = db.GetItem(did);
            } else if (argsType.Equals("Sitecore.Events.SitecoreEventArgs")) { //for local endpoints
                SitecoreEventArgs sargs = (SitecoreEventArgs)args;
                object[] p = sargs.Parameters;
                if (p.Length > 0 && p[0] != null) {
                    Publisher per = (Publisher)p[0];
                    rootItem = per.Options.RootItem;
                }
            }

            if (rootItem == null) {
                Log.Info("HtmlCacheClearer: root item is null", this);
                return;
            }

            SiteContext sc = GetSiteContext(rootItem);
            if (sc == null) {
                Log.Info("HtmlCacheClearer: site context was null", this);
                return;
            }

            HtmlCache htmlCache = CacheManager.GetHtmlCache(sc);
            if (htmlCache != null) {
                ((CustomCache)htmlCache).Clear();
                Log.Info("HtmlCacheClearer: cleared " + sc.Name, this);
            } else {
                Log.Info("HtmlCacheClearer: html cache for " + sc.Name + " was null", this);
            }
        }

        public SiteContext GetSiteContext(Item i) {
            IEnumerable<SiteContext> list = Factory.GetSiteInfoList()
                .Select(a => Factory.GetSite(a.Name))
                .Where(a => !a.StartPath.Equals(string.Empty) && i.Paths.FullPath.Contains(a.StartPath));
            return (list.Any()) ? list.First() : null;
        }
    }
}

??????!!!!!!!!??????!!?..........

What are you looking at me for? Go get that ish. 

I have, for as long as I can remember, always enjoyed storytelling. I've written poetry, comics, produced videos and in more recent years began writing. Some part of it is to help express my emotions, some to turn a joke, some professional development but all were very personal for me. 

For the past few months I've been working on an idea that again was personal to me. That idea is to capture the story behind how many of the developers I know, got their start in programming. I had a very circuitous route and in my opinion got started very late. Others I've met have much more interesting stories and the more people I speak to, the more inspiring it becomes since it appears there really is no one way in. I can't tell you how many developers I've met that majored in English, let alone other even more distant topics. All seem to share a common thread though, and that is passion.

I suppose personally having a roundabout experience led me to find others' stories fascinating and is what has spurred the idea to capture and tell these 'exceptional' stories. Also enjoying my previous experience with producing short videos was why I chose this particular medium. 

So enough with the dithering, I'm starting a YouTube series called "Catching Exceptions" and each episode is a one on one interview delving into the personal stories of the how, what and why of that person's journey into their career. I have no real goal other than to entertain, inform and shine light on what I consider to be some exceptional stories. I'm shooting for releasing about one a month. I've never done this before so I may be in for a rude awakening but sometimes you just have to dive in head first. 

In the mean time, here's a promo video to get a taste of what to expect. Cheers!

I've just released the first episode of Catching Exceptions featuring Nick Wesselman. Enjoy!

Last December, before a soul crushing winter had fully descended on Boston, Sitecore sponsored a posh Meetup at Stoddards pub. While mixing it up, I ended up talking to Rick Cabral about making some t-shirts for the user group. While at the Sitecore Symposium the previous fall, I noticed a few other groups had made their own shirts and I thought it was really cool and as it turns out, I may know a guy who makes t-shirts. So knowing I could supply the backend, I asked Rick what kind of design he thought would be good. I didn't really expect it but his eyes widened and he started talking about how he always envisioned this soviet-era propaganda with the slogan "Join the Sitecore". I almost died laughing because I could totally see it. I didn't need any other explanation and let's be honest, with the goatee, he looks kinda like Vladimir Lenin and would perfectly fit the caricature. And just so you know, as Rick continues to insist, he didn't come up with the idea of immortalizing himself, I did.

Shortly after the meetup I contacted my old, old friend Jay at Antidesigns to produce the design. Working with his team, we went back and forth for few rounds, working from some cruddy source images I pulled off the intarwebs but the image wasn't good enough. It was just too grainy and he wasn't projecting the right message. Here's a few of the early designs:

Eventually I just needed to get a good photo of Rick, so I grabbed him after the user group and he breaks out this World War II helmet, gives this classic pose and everything else fell into place. 

With the design locked down. The next step was producing product but I wasn't sure how to get the funding for the shirts and I didn't want the design to languish so I ponied up the dough myself for the first run. Of course, if you're making a propaganda piece, quite literally, I really didn't think making t-shirts could do it justice; I had to make posters. Talk about getting sidetracked though.

I started with a run of 25 and have since ordered another 15. They're limited edition, artist signed and numbered. They're printed on a parchment type paper and they're absolutely awesome. Rick got the first and I gave him 10 more to dole out as he sees fit for the user group as prizes or whatever. The rest I'm going to be giving away as mementos. I wish I could give one to everyone but they're high quality art and they cost a bit so I couldn't but as owner of the copyright on the design I'm opening it up to the community to use freely (see copyleft).

You can download the vector files here!

Here's the resulting work in all it's glory:

and here's a close up. You can kind-of, sort-of see the grain of the paper... or not:

Anyway, I'm hoping that this makes it easier to raise dinero for the shirts for our user group since it's much easier to explain something that is tangible. 

Also since I'm opening the design to anyone to use or modify, I strongly encourage anyone who wants it, to brand it for their user group. You could even commission the same artist to make some more Sitecore art for your user group or even modules like Powershell, SIM, TDS etc. (wink, nudge). So please hit up Antidesigns and tell them I sent you. They can print on all kinds of stuff and do everything by hand. I'd absolutely love to see what you can come up with. 

Otherwise, long live the #sitecorecommunity!    

I've just released the second episode of Catching Exceptions featuring Sean Holmesby. 

Follow up with Sean here:

Twitter: @seanholmesby
website: www.seanholmesby.com

I know I've been neglecting my blog but it's for a good reason. I had the good fortune to travel to the Netherlands and attend SUGCON 2015 (while meeting a ton of great people). During that trip I undertook the audacious goal of interviewing eight (yes eight!) people in a single day. The first of which is now the third episode of Catching Exceptions featuring Lars Floe Nielsen. 

Follow up with Lars here:

Twitter: @ln_sitecore
website: www.sitecore.net

I've released the fourth episode of Catching Exceptions featuring Pieter Brinkman. He's your inside man, developer advocate and community evangelist at Sitecore!  

Follow up with Pieter here:

Twitter: @pieterbrink123
website: community.sitecore.net

I've released the fifth episode of Catching Exceptions featuring Robbert Hock. If you're looking for a 'Die Hard'
technology partner, you've found the right guy.  

Follow up with Robbert here:

Twitter: @kayeenl
website: kayee.nl