A runtime for Umbraco

June 15, 2015

This is an old archived post, content maybe out of date, links may be broken and layout may be broken.

It seems like an age ago now, but in 2013 I wrote about mapping Umbraco content to POCO and presented on the state of Umbraco and Azure as I saw it at the time.

I also wrote an article called My three circles of Web CMS Nirvana (I was into diagrams involving circles at the time). This article explained why I wanted a runtime for Umbraco and is probably good background reading for this post. But what I haven’t mentioned until now, is that I went away and built the runtime. In fact you are using it now by reading this post.

My blog is edited and deployed using Umbraco - but there is no Umbraco involved in the hosting of this site.

I think I’ve covered the ”why” in the previous post and this is more about the ”how”, but very briefly to recap:

  • Memory use is a concern in hosting on some of the cheaper Azure websites, memory is limited and overuse stops the site - and having an in memory cache often blows those limits.
  • Some of the Umbraco startup time tasks involve building that in memory cache and associated Examine indexes - and in scalable applications launching new instances fast is important.
  • Sometimes for really fast scalable apps that combine editorial structured content and user generated content Umbraco isn’t the right architecture.

On the last point, joining Umbraco content and user generated content is difficult and slow if there is lots of user generated content.

I’m fresh back from Umbraco codegarden which is always inspiring and I’m pleased to see that lots of the ideas that I had around scaling Umbraco in Azure websites are implemented in Umbraco 7.3. I know that there are ideas around a ”new cache” which isn’t a blob of XML in memory - but while we wait for that, I hope what I write here can provide some inspiration.

Part 1

So this is quite a meaty post which I intend to break into parts. And if you have the willpower to read My three circles of Web CMS Nirvana you’ll be astute enough to realise that part 1 is “to have your CMS output a bunch of files to disc, XML, JSON or whatever – but I’d specify that they should be files and not a database.”

I plan to fully rant about how Umbraco shouldn’t have a relational database at all in full detail at a later date.

So this blog runs from JSON on a file system. The folder structure looks like this:

untitled

The Umbraco tree just maps to a folder structure with a content.json file in each folder.

How?

With the implementation of a single interface that runs upon publish.

using Umbraco.Core.Models;

namespace Moriyama.Runtime.Umbraco.Interfaces
{
    public interface IUmbracoContentSerialiser
    {
        void Remove(IContent content);
        void Serialise(IContent content);
    }
}

The task of the implementation is pretty simple. take the content and write it to disc.

The implementation of IUmbracoContentSerialiser hooks into the Umbraco publish, un-publish and delete events and has access to classes providing some other implementations of interfaces - most importantly IContentPathMapper so it knows where to put the content on disc.

namespace Moriyama.Runtime.Interfaces
{
    public interface IContentPathMapper
    {
        string PathForUrl(string url, bool ensure);
        ...

For me the only thing Umbraco should know about is IUmbracoContentSerialiser to keep the separation between CMS and runtime as clean cut as possible.

What does the JSON look like?

It looks like this (bodyText removed):

{
  "Name": "Create an Umbraco document with Perl and Web services",
  "Type": "BlogPost",
  "CreateDate": "2009-01-09T09:01:00",
  "UpdateDate": "2015-01-19T18:41:22",
  "CreatorName": "Darren Ferguson",
  "WriterName": "Darren Ferguson",
  "Ur": "https://reo.speedwagon.me/content/localhost/2009/1/9/create-an-umbraco-document-with-perl-and-web-services",  "RelativeUrl": "/2009/1/9/create-an-umbraco-document-with-perl-and-web-services/",
  "Content": {
    "umbracoUrlAlias": "/create-an-umbraco-document-with-perl-and-web-services",
    "HideInNavigation": true,
    "umbracoInternalRedirectId": "",
    "redirect": "",
    "displayDate": "2009-01-09T09:01:00Z",
    "title": "",
    "shortUrl": "http://bit.ly/gqqMmf",
    "summary": "'Create an Umbraco document with Perl and Web services' - a blog post by Darren Ferguson about document using Web services, media service, Perl, Technology Internet written on 09 January 2009",
    "tags": "document using Web services, media service, Perl, Technology Internet",
    "bodyText": "",
    "commentsDisabled": ""
  },
  "Template": "Post",
  "CacheTime": null,
  "SortOrder": 1,
  "Level": 5
}

The JSON serialisation process removes the Umbraco specific stuff which we don’t use - like evil integer IDs and is easily serialised and de-serialised using NewtonSoft JSON to the following object:

using System;
using System.Collections.Generic;

namespace Moriyama.Runtime.Models
{
    public class RuntimeContentModel
    {
        public string Name { get; set; }
        public string Type { get; set; }

        public DateTime CreateDate { get; set; }
        public DateTime UpdateDate { get; set; }

        public string CreatorName { get; set; }
        public string WriterName { get; set; }

        public string Url { get; set; }
        public string RelativeUrl { get; set; }
        
        public IDictionary<string, object> Content { get; set; }

        public string Template { get; set; }
        
        public DateTime? CacheTime { get; set; }

        public int SortOrder { get; set; }
        public int Level { get; set; }
    }
}

In case you are wondering, we don’t need Integer IDs or GUIDs, the relative URL is a perfectly good unique identifier.

The internals of mapping IContent to **RuntimeContentModel **are based around my article mapping Umbraco content to POCO (and I will share the source for all of this).

One last thing here - IUmbracoContentSerialiser discovers implementations of IUmbracoContentParser with reflection and passes the RuntimeContentModel through them before serialising to disc.

using Moriyama.Runtime.Models;

namespace Moriyama.Runtime.Umbraco.Interfaces
{
    public interface IUmbracoContentParser
    {
        RuntimeContentModel ParseContent(RuntimeContentModel model);
    }
}

An IUmbracoContentParser allows you to resolve and modify Umbraco properties. Here is a trivial implementation that renames umbracoNaviHide to something non Umbraco related - but more common uses would be to turn pickers that pick integer IDs into the relative URLs that I need.

using System.Linq;
using Moriyama.Runtime.Models;
using Moriyama.Runtime.Umbraco.Interfaces;

namespace Moriyama.Runtime.Umbraco.Application.Parser
{
    public class NaviHideUmbracoContentParser : IUmbracoContentParser
    {
        public RuntimeContentModel ParseContent(RuntimeContentModel model)
        {
            var newContent = model.Content.ToDictionary(entry => entry.Key, entry => entry.Value);

            foreach (var property in model.Content)
            {
                if (property.Key != "umbracoNaviHide") continue;

                var v = property.Value;
                newContent.Remove(property.Key);

                var newValue = v.ToString() != "0";
                newContent.Add("HideInNavigation", newValue);
            }

            model.Content = newContent;
            return model;
        }
    }
}

So I think that is more or less it for Part 1. I’ve got a disc full of JSON that I can send anywhere - and I can still use Umbraco as my CMS. I’m giving myself a pat on the back.

In Part 2 - I’ll look at how I can render this content as webpages in Umbraco templates. In part 3 I’ll look at how to deploy this runtime into production without Umbraco, so I’ve truly separated my runtime and my CMS.

For those of you still thinking Why? It is and edge case, definitely.

And I’ll leave you with some code - the implementation of IUmbracoContentSerialiser If you’d like access to the whole source - I’ll put up the URL of the source of this blog on the next post. It is a little embarrassing just now and needs some polish:

using System.Collections.Generic;
using System.Reflection;
using System.Text.RegularExpressions;
using AutoMapper;
using log4net;
using Moriyama.Runtime.Models;
using Moriyama.Runtime.Umbraco.Interfaces;
using Umbraco.Core.Models;
using Umbraco.Web;

namespace Moriyama.Runtime.Umbraco.Application
{
    internal class UmbracoContentSerialiser : IUmbracoContentSerialiser
    {
        private static readonly ILog Logger = LogManager.GetLogger(MethodBase.GetCurrentMethod().DeclaringType);

        private readonly UmbracoHelper _umbracoHelper;
        private readonly IEnumerable _contentParsers;
        
        public UmbracoContentSerialiser(UmbracoHelper umbracoHelper, IEnumerable contentParsers)
        {
            _umbracoHelper = umbracoHelper;
            _contentParsers = contentParsers;
        }

        public void Remove(IContent content)
        {
            var publishedContent = _umbracoHelper.TypedContent(content.Id);

            if(publishedContent != null)
                RuntimeContext.Instance.ContentService.RemoveContent(publishedContent.Url);
        }

        public void Serialise(IContent content)
        {
            var publishedContent = _umbracoHelper.TypedContent(content.Id);


            if (publishedContent == null)
                return;

            var runtimeContent = Mapper.Map(publishedContent);

            runtimeContent.Url = RemovePortFromUrl(publishedContent.UrlWithDomain());
            runtimeContent.RelativeUrl = publishedContent.Url;
            runtimeContent.CacheTime = null;

            runtimeContent.Type = publishedContent.DocumentTypeAlias;

            runtimeContent.Template = publishedContent.GetTemplateAlias();

            runtimeContent.Content = new Dictionary<string, object>();

            foreach (var property in content.Properties)
            {
                if (!runtimeContent.Content.ContainsKey(property.Alias))
                    runtimeContent.Content.Add(property.Alias, property.Value);
            }

            foreach (var contentParser in _contentParsers)
            {
                runtimeContent = contentParser.ParseContent(runtimeContent);
            }
            
            RuntimeContext.Instance.ContentService.AddContent(runtimeContent);
        }

        private string RemovePortFromUrl(string url)
        {
            var rgx = new Regex(@"\:\d+"); // get rid of any port from the URL

            url = rgx.Replace(url, "");
            return url;
        }

    }
}

Comments

Darren - June 15, 2015

Message me if you want the not so pretty code before i tidy it up :)

Jamie Pollock - June 15, 2015

Looks great Darren, I’ve experimented with this concept but it seems you’ve gone a step further and got a live app.

Great work. FYI I called my experiment uLightweight, it made me chuckle at least.

Lee - June 15, 2015

Love this concept. Especially for (As you have mentioned) Azure and memory limitations. I remember Aaron or Shannon doing something similar and writing actual .aspx files to disc (Can’t remember what the name of the package was). Looks like you have taken it a lot further with separation and using JSON files, very interested to read the next posts.

Dan Diplo - June 15, 2015

Totally agree a relational database seems the wrong persistence method. This is really interesting, though it raised two questions in my mind:

  1. Why not just publish to static HTML? Wouldn’t this be even faster and lightweight? Does it have some advantages?
  2. How do you handle more interactive stuff i.e. things that would require controllers in Umbraco, such as form submissions, dynamic search, members etc? Or is that not possible?

Looking forward to the rest of the episodes…

Darren - June 15, 2015

@dan - #2 will probably be answered in the next parts.

But #1 I haven’t needed a purely static HTML site for years, though the interfaces provided for the deployment “episode” would allow that.

David Peck - June 15, 2015

Very smart, though I can’t work out how you can do cross content queries without running into the same memory issues. Examine? A NoSQL DB as a cache in production environments, if you could toggle it much in the same way as you can with a session state server. Role on part 2.

Ben McKean - June 16, 2015

Great post Darren, really interesting stuff. Definitely think a more light weight route is the way to go. Looking forward to the next posts

Petr Snobelt - June 16, 2015

@dan: I create html pages using https://github.com/PetrSnobelt/UmbracoStaticPublish and it works fine for webs without dynamic content

I look forward for future “episode”

WilliamReer - June 18, 2016

Fantastic post.Really looking forward to read more. Will read on… Vayner

About the Author

About Darren

Leave a Comment

Comments are manually moderated and added once reviewed.