processItem Pipeline in Relation to FXM

I’ve talked a lot recently about the processItem pipeline and how it can be extended to automatically associate site visitors to Sitecore profiles without having to tag all of your page items in the content tree, in fact it was the theme of my Sitecore Symposium talk in Orlando.

What I have discovered fairly recently is that FXM (Federated Experience Manager) uses the exact same pipeline to register goal conversions, page events etc. on external sites. If you haven’t heard of FXM before, essentially FXM allows you to do “Sitecore like things” on sites that are not running on Sitecore i.e.

  • Track user activity
  • Inject Sitecore managed content into external sites
  • Personalise content injected from Sitecore on external sites

Not an exhaustive list but you get the point. It’s not a silver bullet however and it’s far from perfect but let’s say you want to use the FXM APIs in a non-website context, let’s say mobile apps, to track user activity and goal conversions or do the same kind of profile mapping that I talk about in my Symposium talk. Well, because FXM uses the processItem pipeline within a page tracking context you absolutely can, even though FXM has no back-end UIs that will allow users to manage tracking in a non-website context.

So how exactly? Well firstly when you create an external site in FXM you end up with a “Domain Matcher” item, which defines mappings between the external sites host name and the FXM site it relates to and language mapping rules. This allows you to tell Sitecore how to intepret FXM requests from external sites with the freedom to define how the external site provides that information, could be in the request headers for example.

fxm-domain-matcher

As you begin to set “filters” for the external site using the FXM version of the experience editor (not available for mobile apps of course), for example where the external site address being tracked equals a specific page then register the goal conversion “subscribed to newsletter”, filter items are created as children of the domain matcher. The goals to register etc. are then stored against the tracking field of the child filter item.

The RegisterEventsForMatchedPagesProcessor within the tracking.matchpages pipeline defined in Sitecore.FXM.config then basically iterates over the child filter items that match the current FXM external site tracking request, creates an instance of ProcessItemArgs and passes it into the processItem pipeline.

fxm-events-processor

So again, it would be a relatively trivial thing to extend either the FXM tracking.matchpages or processItem pipeline to read from a repository of rules based profile map items to associate the current visitor with Sitecore profiles even within a non-Sitecore and non-website context.

I am currently working on a project where we will quite likely take this approach, I will update this post with my findings.

Happy profiling!

Advertisements

Sitecore Symposium 2018 Slides

symposium-2018

I had a great time presenting my talk “Taxonomy & Rules Based Profile Mapping” at this years Sitecore Symposium in Orlando, the best comment I had after the talk was a gentleman who came over after and said “thanks so much, that was one of the best developer talks of the conference so far”, it really validated all the hard work and consideration that goes into these things.

I’ve uploaded the slides from the talk for your reference as a pptx so you have the notes, get your download on:

Taxonomy & Rule Based Profile Mapping

The link to the GitHub repository containing the source code is in the slides but if you just want that you can grab it here. I still need to work on the documentation but it’s coming.

It’s been two years in a row speaking at Symposium (last year was mobile app development with the Sitecore Xamarin SDK), let’s see if I can make it three (wherever it may be)!

Sitecore EntityService with ItemService Style Token Authorization

Sitecore’s ItemService and EntityService have been around for a while now and there is a fair amount of community content out there (and Sitecore’s documentation) that describe the role of the two service layers and how to configure them so I won’t reinvent those wheels here.

What I have recently come to discover however is that the EntityService, whilst providing better support for working with custom business objects and writing custom WebAPI controllers does not support authorization out-of-the-box. The settings in Sitecore.Services.Client.config that relate to authorization all apply to the ItemService only.

What I wanted for my project, was the flexibility of the EntityService with (JWT) token based authorization. Best of both worlds.

Ian Graham wrote an excellent article on enabling token based authorization, which you can read here. The configuration steps mentioned there are all relevant to this scenario so consider them a pre-requisite.

After configuring token based authorization and setting up an EntityService you will immediately notice that you can send a request to your service without a token header and it will return a 200 response and the expected JSON data. So how can we use the out-of-the-box token generation and validation that Sitecore provides with the ItemService within the context of EntityService controllers?

Well first, you need to understand that Sitecore’s implementation uses the class Sitecore.Services.Infrastructure.Sitecore.Security.ConfiguredOrNullTokenProvider, which is instantiated in the constructor of Sitecore.Services.Infrastructure.Sitecore.Security.TokenDelegatingHandler


public TokenDelegatingHandler() : this(new ConfiguredOrNullTokenProvider(new SigningTokenProvider()), new UserService())
{
}

The TokenDelegatingHandler is configured in Sitecore.Services.Client.config


<api>
    <services>
        <configuration type="...">            
            <delegatingHandlers hint="list:AddDelegatingHandler">
                <delegatingHandler desc="TokenDelegatingHandler">Sitecore.Services.Infrastructure.Sitecore.Security.TokenDelegatingHandler, Sitecore.Services.Infrastructure.Sitecore</delegatingHandler>
            </delegatingHandlers>
        </configuration>
    </services>
</api>

The TokenDelegatingHandler uses the following method to validate a token passed in the request header


private void AttemptLoginWithToken(HttpRequestMessage request)
{
    if (request.Headers.Contains("token"))
    {
        ITokenValidationResult tokenValidationResult = this._tokenProvider.ValidateToken(request.Headers.GetValues("token").FirstOrDefault<string>());
        
        if (tokenValidationResult.IsValid)
        {
            if ((
                from c in tokenValidationResult.Claims
                where c.Type == "User"
                select c).Count<Claim>() == 1)
            {
                string value = tokenValidationResult.Claims.First<Claim>((Claim c) => c.Type == "User").Value;
                this._userService.SwitchToUser(value);
            }
        }
    }
}

What’s important here is that an instance of HttpRequestMessage is passed into this method.

Now let’s create a custom authorize attribute that inherits from System.Web.Http.AuthorizeAttribute. In this class override the IsAuthorized method, which is passed an instance of HttpActionContext and let’s give it a constructor that creates a new privately available instance of ConfiguredOrNullTokenProvider just like the TokenDelegatingHandler (except we don’t also need the UserService, we just want to know if the token is valid or not).

The HttpActionContext instance passed into the IsAuthorized method gives us access to the HttpRequestMessage (which the TokenDelegatingHandler uses to locate the token header, so we can do the same. Putting it all together


public class TokenAuthorizeAttribute : AuthorizeAttribute
{
    private readonly ITokenProvider _tokenProvider;

    public TokenAuthorizeAttribute() : this (new ConfiguredOrNullTokenProvider(new SigningTokenProvider()))
    {

    }

    public TokenAuthorizeAttribute(ITokenProvider tokenProvider)
    {
        _tokenProvider = tokenProvider ?? throw new ArgumentNullException("tokenProvider");
    }

    protected override bool IsAuthorized(HttpActionContext actionContext)
    {
        if (!actionContext.Request.Headers.Contains("token"))
            return false;

        var tokenValidationResult = _tokenProvider.ValidateToken(actionContext.Request.Headers.GetValues("token").FirstOrDefault());           

        return tokenValidationResult.IsValid;
    }
}

Now we simply decorate our entity service controller with our custom token authorization attribute


[ServicesController]
[EnableCors(origins: "*", headers: "*", methods: "GET")]
[TokenAuthorize]
public class ProductController : Sitecore.Services.Infrastructure.Sitecore.Services.EntityService<Product>
{

}

Now you can request tokens the same way you do with the ItemService:

Request Sitecore token using Postman

And pass the value of the token response back to Sitecore in the token header when making requests to your entity service controllers or get a 403 response.

As highlighted on Sitecore Stack Exchange you will get a 500 server error response if you pass a token that has expired (expiry is 20 minutes by default but configurable in the setting Sitecore.Services.Token.Authorization.Timeout), well certainly in 8.2 Update 2 and Update 3 you will, I haven’t tested other versions. If you don’t care about that, you are done here.

If however, you do want to get a proper 403 response for expired tokens (like me and the OP on Stack Exchange) read on.

To make this error go away we have to (unfortunately) do some slightly dirty things. First we need to replace the default implementation of TokenDelegatingHandler, which is straight forward to do because it’s just configuration


<api>
    <services>
        <configuration>
            <delegatingHandlers hint="list:AddDelegatingHandler">
                <patch:delete />
            </delegatingHandlers>
            <delegatingHandlers hint="list:AddDelegatingHandler">
                <delegatingHandler desc="TokenDelegatingHandler">MyHandler, MyAssembly.dll</delegatingHandler>
            </delegatingHandlers>
        </configuration>
    </services>
</api>

In our custom handler we want to replace the default SigningTokenProvider with our own implementation in two of the constructors


public class MyHandler : TokenDelegatingHandler
{
    public MyHandler()
        : base(new ConfiguredOrNullTokenProvider(new MySigningTokenProvider()), new UserService())
    {

    }

    protected MyHandler(HttpMessageHandler innerHandler)
        : base(innerHandler, new ConfiguredOrNullTokenProvider(new MySigningTokenProvider()), new UserService())
    {

    }
}

This is where it gets gory, yes you guessed it, no virtual methods, private members etc so we have to copy paste a bunch of Sitecore’s code and replace the bit responsible for the 500 error which as the thread on Stack Exchange explains is in the logging of SecurityTokenInvalidLifetimeException exceptions raised. I’ve thrown up a gist so you don’t have to go through the pain of tidying it all up.

That’s it. Sitecore’s EntityService with ItemService style token authorization and proper 403 responses for expired tokens.

Happy… securing.

Sitecore & Xamarin for Mobile Apps

I had the pleasure of presenting this topic at Sitecore Symposium this year in Las Vegas. The aim of the topic was to demonstrate the steps required to get started with Xamarin development in conjunction with Sitecore’s Mobile SDK.

I have turned this presentation into an article, which you can find here:

https://www.valtech.com/blog/sitecore–xamarin-for-mobile-apps/

You can also download the slides from the presentation here:

Everyone gets an app!

Sitecore Quick Tip: Getting and Setting SafeDictionary

Sitecore’s SafeDictionary is used throughout Sitecore’s libraries and the purpose of it (as opposed to using Dictionary) is to allow dictionary[key] without exceptions, instead returning default(T).

To read values into a SafeDictionary from a Sitecore field that uses a query string internally (validator parameters, rendering parameters, name value field etc.) you can use:


var parameters = Sitecore.Web.WebUtil.ParseQueryString(item["Field Name"]);

Then of course you can modify the SafeDictionary:


if (!parameters.ContainsKey("foo"))
{
    parameters.Add("foo", "bar");
}

And writing the SafeDictionary back to the field:


item["Field Name"] = string.Join("&", parameters.Select(x => $"{System.Web.HttpUtility.UrlEncode(x.Key)}={System.Web.HttpUtility.UrlEncode(x.Value)}"));

Sitecore Sunday Pt 4: Conditional (Rules Based) Field Validators

Sitecore’s OOTB field validators are used on pretty much every Sitecore implementation I’ve seen in the last 10 years, which makes it a pretty useful feature! However there are times when there is a little more context surrounding whether or not a particular field is valid.

Thankfully Sitecore makes it super simple to create custom field validators where we have reign to do pretty much whatever we want and there have been some interesting ones put out there for example this image width, height and aspect ratio validator by Brian Pedersen.

I will be calling this approach IFTTV (If This Then Valid).

If This Then Valid

Sorry.

To setup field validator items that use the rules engine we first of course need a template with the necessary validation fields. Sitecore actually already provides this template:

/sitecore/templates/System/Validation/Rules Validation Rule

However this template is intended for use with item level validation that reads from rules under the item:

/sitecore/system/Settings/Rules/Validation Rules/Rules

I feel this approach is disconnected from the management of field validation rules under:

/sitecore/system/Settings/Validation Rules/Field Rules

But we can still take advantage of the template. The first step is to create a template that inherits from the Rules Validation Rule template and specifies our field level rules implementation in the type field.

If This Then Valid Template

The implementation is fairly straight forward:


using Sitecore.Data.Validators;
using Sitecore.Rules;
using Sitecore.Rules.Validators;
using System.Runtime.Serialization;

public class IfThisThenValid : StandardValidator
{
    public override string Name => "If This Then Required";

    public IfThisThenValid()
    {
    }

    public IfThisThenValid(SerializationInfo info, StreamingContext context) : base(info, context)
    {
    }

    protected override ValidatorResult Evaluate()
    {
        var item = GetItem();

        if (item == null)
            return ValidatorResult.Valid;

        var ruleContext = new ValidatorsRuleContext
        {
            Item = item,
            Validator = this,
            Result = ValidatorResult.Valid,
            Text = string.Empty
        };

        var validatorItem = item.Database.GetItem(ValidatorID);

        if (validatorItem == null)
            return ValidatorResult.Valid;

        var rules = RuleFactory.GetRules<ValidatorsRuleContext>(new[] { validatorItem }, "Rule");

        if (rules == null)
            return ValidatorResult.Valid;

        rules.Run(ruleContext);

        if (ruleContext.Result == ValidatorResult.Valid)
            return ValidatorResult.Valid;

        base.Text = ruleContext.Text;

        return base.GetFailedResult(ruleContext.Result);
    }

    protected override ValidatorResult GetMaxValidatorResult()
    {
        // Critical or above (Fatal) is important for experience editor support

        return base.GetFailedResult(ValidatorResult.CriticalError);
    }
}

To summarise:

  • Inherit from Sitecore’s StandardValidator
  • Get the item being validated (rules are run against this as the context item)
  • Get the validator item (this is where our rules live)
  • Get the rules in the validator item and run them against the context item using a ValidatorsRuleContext (which returns a ValidationResult). This is where the difference lies from Sitecore’s standard implementation*, which evaluates the context item against the rules under: /sitecore/system/Settings/Rules/Validation Rules/Rules
  • Return the result

*Sitecore.Data.Validators.ItemValidators.ValidationRulesValidator,Sitecore.Kernel

Then I will create a folder under /sitecore/system/Settings/Validation Rules/Field Rules and an insert options rule so we can create new rules based field validators and organise them in folders:

If This Then Valid Insert Options

As an example I added two fields to the Sample Item template, Sitecore Image and External Image, the validation scenario is at least one of them must be provided. Here’s our rule based validator:

If This Then Valid Rule

Which is associated with our fields on the sample item:

If This Then Valid Field

And here’s it in action:

If This Then Valid Usage

And that’s it, pretty straight forward but a ton of uses, especially considering the extensibility of the rules engine.

Happy validating!

Sitecore Sunday Pt 3: Conditional (Rules Based) Goal Registration

Goals in Sitecore allow us to register the fact that a site visitor has performed a specific action on the site and more importantly that that action has engagement value associated with it. Actions can vary but typically fall into one of two buckets, significant page views (end of conversion funnel pages) and on-page events (interacting with a map, playing a video etc.). For more on the importance of engagement value as a metric take a look at this guide by Sitecore’s SBOS (Sitecore Business Optimisation Services) team.

This post tackles more advanced scenarios where a certain pre-condition must be satisfied before considering a page view or on-page event as legitimate engagement, to put it another way, we want to qualify the action before registering the conversion. The example I will use is site search related but the concept and approach can easily be extended to cover other scenarios. The scenario is this, we want to register the “Site Search Page View” goal when a visitor views a search results page but not when the user subsequently pages through the search results as this is considered an extension of behavior but ultimately part of the original conversion.

The “Site Search Page View” goal looks like this.

Sample Internal Search Goal Item

On inspection we can see that the goal has a rules field. This field however does not represent a pre-condition that must be true before the goal is registered, it allows us to trigger further actions after the goal is registered if the conditions of the rules field are true. This could have some interesting uses but doesn’t solve the problem we are trying to solve here.

The solution to our problem once again lies in the processItem pipeline, defined in Sitecore.Analytics.config.


<processItem>
  <processor type="Sitecore.Analytics.Pipelines.ProcessItem.CollectParameters,Sitecore.Analytics" />
  <processor type="Sitecore.Analytics.Pipelines.ProcessItem.TriggerCampaigns,Sitecore.Analytics" />
  <processor type="Sitecore.Analytics.Pipelines.ProcessItem.RegisterPageEvents,Sitecore.Analytics" />
  <processor type="Sitecore.Analytics.Pipelines.ProcessItem.ProcessProfiles, Sitecore.Analytics" />
</processItem>

This is an extremely useful pipeline, which I also make use of in the post Sitecore Sunday Pt 2: Programmtic Profile Association. When goals are associated with page items in the traditional way (select the page and select Analyze > Goals), this information is stored in the pages __Tracking field as xml, just as associated profiles are.

The CollectParameters processor checks for the existence of the __Tracking field on the context item (or the context items template standard values) and if it is not null adds the tracking field to the TrackingParameters property of the ProcessItemArgs passed into Process method of the processor (a bit of a mouthful yes).


public class CollectParameters : ProcessItemProcessor
{
    public override void Process(ProcessItemArgs args)
    {
        // code removed for brevity

        if (!string.IsNullOrEmpty(args.Item["__Tracking"]))
        {
            Field item = args.Item.Fields["__Tracking"];

            if (item != null)
            {
                args.TrackingParameters.Add(new TrackingField(item));
            }
        }
    }
}

The RegisterPageEvents processor further down in the pipeline uses the TrackingParameters property to collect up all the goals associated with the current context (request) and registers them against the current page on the Interaction (session) object of the current contacts xDb record.

The trick now is to add additional tracking fields to the TrackingParameters property if certain conditions are true. To do this we will add an additional processor to the pipeline after the CollectParameters processor (to preserve standard functionality) and we’ll call it CollectConditionalParameters.


<pipelines>
  <processItem>
    <processor
      type="Sitecore.Feature.ConditionalGoals.Pipelines.ProcessItem.CollectConditionalParameters, Sitecore.Feature.ConditionalGoals"
      patch:after="*[@type='Sitecore.Analytics.Pipelines.ProcessItem.CollectParameters,Sitecore.Analytics']" />
  </processItem>
</pipelines>

The code in this processor looks like this:


public class CollectConditionalParameters : ProcessItemProcessor
{
    public override void Process(ProcessItemArgs args)
    {
        var goals = ConditionalGoalsRepositoryFactory.Repository.Goals();

        foreach (var goal in goals.Where(x => x.IsMatch()))
        {
            args.TrackingParameters.Add(new TrackingField(goal.Fields["__Tracking"]));
        }
    }
}

To summarise before going into more detail:

  1. A collection of conditional goals is retrieved from a repository (obvious I know, detail to follow).
  2. For each conditional goal that is considered a match for the current context (based on its rules), add the tracking field on the conditional goal to the TrackingParameters property, which will be used later on in the pipeline by the RegisterPageEvents processor

The conditional goals repository gets a collection of items (separate from the goal items in the Marketing Control Panel and scoped to specific sites to support multi-tenancy) based on the “Conditional Goal” template. This template has three fields:

  1. Applies to Templates
  2. Rules
  3. Attributes To Register

The applies to templates field allows the author to select which page templates the goal applies to or if it applies to any page type, can be left blank. The rules field allows the author to set the condition(s) that must be satisfied for the goal to be registered. You’re probably thinking “The rules engine has conditions for the template that the context item is based on, why do we need the applies to templates field?”. Whilst this is a correct assertion, traditionally goals are assigned to specific pages or templates and marketers more often than not associate specific goals with specific page types, it felt an important enough condition to pull it out and pay special attention to it.

Finally the attributes to register field is a field that uses the field type “Tracking” and allows the author to select the specific goal that should be registered if all conditions are true.

Conditional Goal Item

One thing I have noticed about the tracking field is that regardless of the values selected the field always displays “Nothing has been set”, I have opened a ticket with Sitecore support, I will update this post when I get a response.

Sitecore Support Response: This is a system field, when selections are made it modifies the __Tracking field (which we know). The text “Nothing has been set” is hard coded. Not considered a bug.

So that is really all there is to it, a simple yet powerful way to register page view goals but only when specific conditions are met, powered by the Sitecore rules engine and processed in the processItem pipeline.