fixing WFFM tracking field

After installing a the new version of WFFM (8.2) we were seeing broken links on existing form items. Thanks to the lovely broken links gutter, to guide us to this mistake.

After some googling, as any self respecting developer does when facing a problem for the first time, I found this blog post. It seems that the custom XML being stored as the raw value of the Tracking field has had an update in the newer WFFM versions. As suggested in the blog, one way to fix this is by opening the attributes and goals of the tracking field and clicking OK, without making any changes. This updates the XML and as if by magic the gutter disappears.

So what changed in the raw values? Well in the new version, the events now have an ID attribute. Here is an example of the raw values before the “fix”:

1
2
3
4
5
6
7
<tracking>  
<event name="Field Changed" />
<event name="Field Completed" />
<event name="Field Not Completed" />
<event name="Field Out of Boundary" />
<event name="Invalid Field Syntax" />
</tracking>

And the raw values afer the “fix”:

1
2
3
4
5
6
7
<tracking>
<event id="{7E86B2F5-ACEC-4C60-8922-4EB5AE5D9874}" name="Field Not Completed" />
<event id="{F3D7B20C-675C-4707-84CC-5E5B4481B0EE}" name="Field Out of Boundary" />
<event id="{844BBD40-91F6-42CE-8823-5EA4D089ECA2}" name="Invalid Field Syntax" />
<event id="{AA3AE715-E87D-4B4D-80C7-4290546F770F}" name="Field Changed" />
<event id="{F0113A93-570A-4F69-8C7C-BA08037D1E34}" name="Field Completed" />
</tracking>

Since I didn’t feel like clicking a thousand times to update 400+ forms (older versions are forms too!) I wrote an admin page to fix it. Here’s the code I used. It might not be the cleanest, but it gets the job done. Just save it as an .ASPX file in your site under /sitecore/admin and you are good to go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
<%@ Page Language="C#" %>
<%@ Import Namespace="System" %>
<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="System.Linq" %>
<%@ Import Namespace="Sitecore.ContentSearch" %>
<%@ Import Namespace="Sitecore.ContentSearch.SearchTypes" %>
<%@ Import Namespace="Sitecore.Data" %>
<%@ Import Namespace="Sitecore.Data.Items" %>
<script runat="server" type="text/C#">
private bool authorized = false;
private string UnauthorizedMessage = "You are not authorized to view this page";
protected void Page_Load(object sender, EventArgs e)
{

if (!(true
//&& Request.IsSecureConnection // Check for HTTPS
&& Request.UserHostAddress == "127.0.0.1"
&& Sitecore.Context.IsLoggedIn
&& Sitecore.Context.IsAdministrator
//&& Sitecore.Context.User.IsInRole(@"sitecore\Developer") //check for specific role
))
{
Response.StatusCode = (int) System.Net.HttpStatusCode.Forbidden;
Response.Write(UnauthorizedMessage);
Response.TrySkipIisCustomErrors = true;
Response.End();
return;
}
else
{
authorized = true;
}

}

protected void FixFields_OnClick(object sender, EventArgs e)
{

if (!authorized)
{
output.InnerHtml = string.Format("<div style=\"color:red;\">{0}", UnauthorizedMessage);
return;
}
ID formTemplateId = new ID("{FFB1DA32-2764-47DB-83B0-95B843546A7E}");
ID contenTreeId = new ID("{0DE95AE4-41AB-4D01-9EB0-67441B7C2450}");
ISearchIndex index = ContentSearchManager.GetIndex("sitecore_master_index");
string TrackingFieldName = "__tracking";
int changeCount = 0;
int errorCount = 0;
output.InnerHtml += "<div>Starting update of fields</div>";

Dictionary<string, string> replacements = GetReplacements();
List<Item> indexResults = null;
using (IProviderSearchContext context = index.CreateSearchContext())
{
indexResults = context.GetQueryable<SearchResultItem>()
.Where(x => x.TemplateId == formTemplateId
&& x.Paths.Contains(contenTreeId))
.Select(x => x.GetItem()).ToList();

indexResults = indexResults.Where(x => x != null).Where(x => x.Fields[TrackingFieldName] != null).ToList();
if (!indexResults.Any())
{
output.InnerHtml += "<div>No Form items found to modify. </div>";
return;
}

foreach (Item item in indexResults)
{
try
{
string value = item.Fields[TrackingFieldName].Value;
foreach (var dictionaryItem in replacements)
{
value = value.Replace(dictionaryItem.Key, dictionaryItem.Value);
}

item.Editing.BeginEdit();
using (new EditContext(item))
{
item.Fields[TrackingFieldName].Value = value;
output.InnerHtml += string.Format("<div>Item - {0} - changed</div>", item.Name);
}
item.Editing.EndEdit();
changeCount++;
}
catch (Exception ex)
{
output.InnerHtml += string.Format("<div>Item - {0} - {1}</div>", item.Name, ex.Message);
errorCount++;
}
}
}
output.InnerHtml += string.Format("<div>Update of tracking fields finished. {0} items modified, {1} errors ocurred. </div>", changeCount, errorCount);

}

private Dictionary<string, string> GetReplacements()
{
Dictionary<string, string> replacements = new Dictionary<string, string>();
replacements.Add("<event name=\"Field Changed\" />", "<event id=\"{AA3AE715-E87D-4B4D-80C7-4290546F770F}\" name=\"Field Changed\" />");
replacements.Add("<event name=\"Field Completed\" />", "<event id=\"{F0113A93-570A-4F69-8C7C-BA08037D1E34}\" name=\"Field Completed\" />");
replacements.Add("<event name=\"Field Not Completed\" />", "<event id=\"{7E86B2F5-ACEC-4C60-8922-4EB5AE5D9874}\" name=\"Field Not Completed\" />");
replacements.Add("<event name=\"Field Out of Boundary\" />", "<event id=\"{F3D7B20C-675C-4707-84CC-5E5B4481B0EE}\" name=\"Field Out of Boundary\" />");
replacements.Add("<event name=\"Invalid Field Syntax\" />", "<event id=\"{844BBD40-91F6-42CE-8823-5EA4D089ECA2}\" name=\"Invalid Field Syntax\" />");

return replacements;
}
</script>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
<title>WFFM 8.2 fix tracking field</title>
</head>
<body>
<form id="trackingUpdater" runat="server">
<h3>Update tracking field for WFFM form items</h3>
<div style="margin-left:15px;">
Click the button below to fix the Tracking field for WFFM forms. This will add ID attributes to the selected items in this field.
Modify the dictionary to replace more/less as needed.
<br/><br/>
<asp:Button runat="server" ID="FixFields" OnClick="FixFields_OnClick" Text="Update form tracking field" />
</div>
<div id="output" runat="server" style="width: 1000px; height: 500px; overflow-scrolling: auto;">
</div>
</form>
</body>
</html>

The dictionary can be easily expanded in case you selected any other goals or attributes in this field. I only added the ones that I had selected in my forms.

upgrade-sitecore 6.5 to 8.2 - part 3

Reality check

In part 2 I talked about manually upgrading 6.5 to 7.2 and then using the Express Migration Tool, to take it to 8.2. Fortunately, by the time I got started, 6.6 was already supported by EMT.

Here are the high level steps I took to get to 8.2:

  • Made backup of site and databases
  • Updated config files to 6.6 in VS solution
  • Updated dll references in solution to new sitecore dll versions
  • Updated dll references to 3rd party modules
  • Referenced new version of HTMLAgilityPack

The third step was the most annoying here. I had to download the source code of the modules, replace the sitecore references and recompile. Most of them worked with minor tweaking here and there, so I could add them back into my VS solution. The project also heavily relied on 3rd party implementations of the company that originally developed the solution. This included (but was certainly not limited to) a custom buckets setup that didn’t really work well on a split CM and CD setup. I removed all of the references to their dll’s and started commenting out code until my project would build.

After this was done I was finally running on Sitecore 6.6 with a semi-disabled custom solution. Time for the EMT to do it’s thing…. and it did. after a few hours my site was upgraded to Sitecore 8.2. This meant it was once again time to update my sitecore references, comment out or modify code specific to 6.6 and get my VS solution to build again.

Once that was done I had a running 8.2 solution and a very ugly website with very few functionality. Time to bucle down and deep dive into the code, fixing commented out code and rewriting parts that were bugging me.

Deciding what to keep, what to change and what to throw out and rebuild is the biggest challenge in a project like this. Unfortunately I can’t give you any clear answers here but I would suggest to change only what’s needed.

I started with adding code generation for templates and a few item locations because I did not want to keep updating the existing POCO classes and “FieldInfo” classes. Hard coded strings give me nightmares, so I’ll let TDS worry about that and generate some code for me. After that I changed the way the 3rd party had implemented repositories (aka Search), configuration (which are items with a ton of fields) and dictionary (again custom).

Since this is now an 8.2 I thought I’d get on board with Helix and made these parts as foundations and/or features. Now if you’re reading this, you’re in the same boat as me so don’t do what I did next. I made a nice foundation for the base functionality of the repositories and started to move the concrete ones into helix features.

Turns out these existing repositories liked to keep their POCO classes close and the FieldInfo classes and they like their own POCO’s and FieldInfo’s and … You can guess the rest. after a few days of messing about, I had around 40 foundations with a lot of references between them. It may not be the best setup, but at least it provides a better view on what is in this project.

A note on templates

A problem that seems to occur with sitecore upgrades is the template inheritance tab being broken. There is usually a YSOD pointing to the fieldtype.

The easiest albeit time consuming method of fixing this is opening the template that has the bug (since not all of them may be affected). Switch on Raw Values and copy the contents of the base templates field to your favourite text editor.

I’m sure you can guess where this is going, open the templates belonging to these templates and set field types for those fields that have none selected. When you’ve gone through all of them the error should be gone. In my case there were some field sections on the standard template that were causing the issue. The following fields had no field type:

  • Appearance section : **_Preview** field
  • Layout section : **_Controller** field
  • Tasks : **_Archive Version date** field

In case this didn’t fix your problem there was an old bug in the sitecore 6-ish versions. In that case it was the _Owner field. Switch to the core database. open your content editor and select /sitecore/templates/System/Templates/Sections/Security Finally assign a “Single-Line Text” field type to the __Owner field. And that’s the end of that YSOD.

upgrade sitecore 6.5 to 8.2 - part 2

On second thought…

In part 1 I described my temporary plan of attack for upgrading a Sitecore 6.5 project to 8.2. After a nice discussion on the Sitecore community slack I revised my initial plan. Basically I’m going to attach the existing databases to a vanilla site setup and upgrade. Separately I’ll upgrade the code and config to work with a 8.2 site. Finally the two will be linked, resulting in a perfect 8.2 project… or at least that’s the idea and it will have an old(er) codebase.

As for the code part only necessary changes will be made to get everything up and running. meaning changing or upgrading modules and rewriting code if needed. Sadly that means a lot of sheer UI and webforms code will remain for now.

The preparation plan will be mostly the same so I’ll leave it out in this post. The upgrade strategy has been changed and is now more detailed.

Sitecore database upgrade

  1. Stop development
  2. Clear publishing and event queues. set longer timeouts in web.config
  3. Shrink and rebuild db indexes
  4. Update all environments with production content (take care not to loose not deployed development)
  5. Link to vanilla site installations
  6. Upgrade to version 7.2 (or whichever version is supported by then)
  7. Use Express Migration Tool to upgrade to sitecore version 8.2
  8. Backup analysis and upgrade reports and logs

Steps for upgrading new site to next version

  1. Backup master/core/web/analytics databases of previous version
  2. Restore backups to DB’s with new version number
  3. Change app_config/connectionstrings.config to point to new databases (which are still in older version)
  4. Upgrade databases to new version

Refer to my post on bulk duplicating databases to see how I make the backups and restore duplicates.

Code / file upgrade

  1. Modify/upgrade config files to Sitecore version 8.2.
  2. Refactor code for Sitecore 8.2, commenting out all broken (third party or custom) code.
  3. Build/deploy this minimal version.
  4. Use the vanilla site versions until 8.2 is reached
  5. After upgrading databases, start deploying (working parts of) the code
  6. Gradually fix the code and update/replace the modules as needed
  7. Start updating the broken code / modules.
  8. Deploy version 8.2 (without modules) to staging or whatever temporary QA site
  9. Have key users verify the minimal 8.2 version on staging or whatever temporary QA site
  10. Build and deploy upgraded code
  11. Have key users retest entire site.

Steps for updating code to 8.2 version

  1. Branch code
  2. Modify publish settings and other references to site -> vanilla version 8.2
  3. Change all references for Sitecore version 6.5 to version 8.2
  4. Build, bugfix and deploy to site version 8.2
  5. Run (smoke) tests on site 8.2 (still with vanilla db)

Link upgraded databases to upgraded site files

  1. Backup Site and upgraded databases
  2. Change connection strings to upgraded database
  3. Test, bugfix and retest