Using export configuration files in Orchard Import / Export

In the more recent versions of Orchard (I think it came in in 1.9) the export options available at Import/Export became much more powerful. One of the underused features (thanks to a lack of documentation) is "Upload a configuration file with recipe steps to execute" option.

With this option selected, you can upload an xml file that defines which content types and other custom export features you wish to export with your site, allowing you a repeatable process, and avoiding manual mistakes.

Here is an example of such a file.



Note that "PagewithPromotion" is a custom content type that has been added to the site in this example. Unfortunately currently, there is only "inclusive" syntax it appears for listing which content types to export. I'm going to raise an issue for adding "exclude" syntax so that it is easier to say "all content types except bob".

I've constructed this template by looking through the source code. The example works, but there are probably also options I've not spotted, so feel free to explore the code further!

Serializing Enums to String in MVC6

Often you will want to change the default serialization of your API responses, to ensure that enums are reflected as strings, not integers. Why? Well, which is more self documenting?

{
  "Status": 1,
  "Message": "things are on fire"
}

or

{
  "Status": "Error",
  "Message": "things are on fire"
}

Fortunately this is really easy in MVC6. The default code uses the ever populate JSON.NET, and it's options are easily exposed. Note that this code example is against RC1, so should actually be correct going forward (the syntax has changed many times across vnext). I'm also assuming you have created your site using the template application, rather than creating an empty ASP.NET application.

Inside the "ConfigureServices" method find

services.AddMvc();


and replace it with

services.AddMvc().AddJsonOptions(options =>
{
    options.SerializerSettings.Converters.Add(new StringEnumConverter());
});

As you can see, you should be able to get at all the other json.net options from here too.

Lamda expressions in unit testing are hard

So, it turns out that Lamda expressions are not great for unit testing. A recent example of code that I wanted to unit test:

 public async Task RegisterIndexAsync<T>(IFoundocIndex<T> index, CancellationToken cancellationToken)  
     {  
       //.....  
       await _fdbStorageProvider.ReadWriteAsync(async transaction =>  
       {  
         var indexDefinitionState = await _indexProvider.GetIndexDefinitionStateFromStore(transaction, index).ConfigureAwait(false);  
         if (!indexDefinitionState.Exists || indexDefinitionState.Changed)  
         {  
           await _indexProvider.PersistIndexToStore(transaction, index).ConfigureAwait(false);  
           var documentCount = await _documentProvider.Count<T>(transaction).ConfigureAwait(false);  
           if (documentCount > 0)  
           {  
             Trace.WriteLine(documentCount + " items found in collection");  
             rebuildIndex = true;  
           }  
         }  
       }).ConfigureAwait(false);  
       if (rebuildIndex)  
       {  
         Trace.WriteLine("Rebuilding Index: " + index.Name);  
         await RebuildIndexAsync(index, cancellationToken).ConfigureAwait(false);  
       }  
       Trace.WriteLine(String.Format("Not Rebuilding Index: {0}.", index.Name));  
     }  
     public async Task RebuildIndexAsync<T>(IFoundocIndex<T> index, CancellationToken cancellationToken)  
     {  
       using (var queue = new BlockingCollection<IEnumerable<T>>(_settings.MaxBatchesInIndexQueue))  
       {  
         _batchEntityProvider.GetBatches(queue, _settings.MaxIndexBatchSize).ConfigureAwait(false);//deliberately not awaiting this  
         await _batchConsumer.Consume(queue, cancellationToken, async batch => await ConsumeWorkQueue(batch));  
       }  
     }  

Now of course, I'm unit testing an implementation call to RegisterIndexAsync<T>(IFoundocIndex<T> index, CancellationToken cancellationToken) but I also want to verify that in this test, that my index was not rebuilt. Normally you could do this by mocking (for example, using the amazing Moq and verifying the number of calls to_batchEntityProvider.GetBatches but here there is a complication.

In this example you would need to use a Setup operation on _fdbStorageProvider.ReadWriteAsync that would supply the entire of the lamda expression as its setup. Essentially - you would need to know and express the code for this function in your unit test setup. Your unit test becomes essentially "ensure that what the code does is what the code does" - and this is not right.

It is also extremely hard to do due to the way lamda expressions compile - the same resulting code will compile to a different object - so they are never going to the same object in your Moq setup call.

Looking more deeply into this example you could validly say that I shouldn't care about verifying whether an internal operation is called. All I should be worried about are external results right?

Quite possibly true in this case - if I cannot observe the impact through external interfaces its probably not worth knowing right? Except that I need to know if this method is rebuilding an index unnecessarily. If it does, there will be no observable difference - the index would be the same before and after, the only difference would be the time taken on larger indexes - something you can't identify in a unit test, and using time taken as a part of a test is a lousy idea anyway.

For this specific test, I think it is time to head back to integration tests, which leads to the future problem of - how do you do the equivalent of "Verify" when you are not mocking your classes. That's for another time.

How to Setup Async and Task Return methods with Moq 4.2

Moq 4.2 comes with a couple of nice changes that I hadn't noticed (and they are extension methods, so you might continue to miss them)

The main benefit is allowing you to change from writing

 _mock.Setup(m => m.GetStateAsync(It.IsAny<Profile>()))   
 .Returns(Task.FromResult(new IndexDefinitionState(true, true)));  

to writing

  _mock.Setup(m=>m.GetStateFromStore(It.IsAny<Profile>()))  
    .ReturnsAsync(new IndexDefinitionState(true, true));  

...which is just that little bit easier to manage (especially when it is a more complex return type than the example above), but it also allows methods with return type Task to work without further setup it seems. Both are extremely useful for the Async-first API I'm working on.

From the release notes for Moq 4.2

• Improved support for async APIs by making default value a completed task
• Added support for async Returns and Throws
• Improved mock invocation sequence testing

All great stuff. I really couldn't do without Moq - a long time back it was the thing that made me realise that unit testing was actually viable.

The battle for the internet is back on....

Battle for the Net | What is Net Neutrality

Thoughts on The Phoenix Project

I finally got round to reading The Phoenix Project last weekend. I know right? It's about time. I thought I'd share a few thoughts, as I think it's a great book and well worth a read for anyone in a business delivering products depending on IT (hint, nearly every business of any size these days depends on successful IT supporting the core business functions).

The Phoenix Project is told as a novel about the recovery story of a business who who used to be the best widget maker in the world, and are now being pounded on by a faster, more agile, hungrier up-and-comer. We follow Bill, who is promoted into the seventh circle of hell - VP of IT Operations, with only 90 days until the release of the-mother-of-all-projects which is vital to the companies survival. It's failing hard, VPs are falling left and right, every meeting looks like Game of Thrones and every piece of the puzzle seems to depend on one engineer, Brent, who's time is more oversubscribed than a year 2000 dotcom IPO.

Before I go any further - I cannot recommend enough the concept of getting your message through as a novel, rather than a text book. There are other books that take this similar approach, The Five Dysfunctions of a Team is another one I love, and for the same reason. It turns dry, bullet point material, and turns it into a "what happens next" adventure. You are with the protagonist on whether they solve their problems, and your brain is following the same steps as they do all the way through. A fiction is worth a thousand bullet points you might say. Consider them "text books with only extremely coherent examples". I read The Phoenix project in two days. I can't remember the last time I read a Terry Pratchett so fast.

There are some key messages to take from it. I want to avoid explaining the entire story of the book, because half of it's power is you working the problem yourself.

The first is actually not one really focused on by the book explicitly in its "lessons", but is worth learning: While development, operations, sales, marketing, "products" etc are all sniping at each other and seeing all other departments as "getting in the way of real work", you are in a pretty bad spot. Maybe a more positive assertion is: All teams need to work together with the vision that they are all responsible for delivering the companies core product

The second is more explicit: Understand the definition of work and from that understand what things your team actually works on and prioritise it. The book lists four, but you could easily make it two; planned work, and unplanned work. The book splits planned into business project, internal project (e.g. infrastructure upgrade) and changes (e.g. production db schema update). The obvious question is this - if you don't know all the work that your team are being asked to do, or where it comes from, how can you possibly ensure that you prioritise it correctly?

The next is a classic from lean, and is well developed in books such as The Toyota Way, and The Lean Startup. Work in progress kills productivity. In IT terms, anything that's started, but not working correctly in production is useless to the business. It is money spent and no return gained. When stock analysts look at companies they are interested in investing in, they look at how well they convert raised capital into further gains, and so should we as IT professionals. It ties in very closely to one of my own favourite mantras on "the definition of done", which I firmly believe can only be interpreted as working bug free as desired by the client or business in the production environment. Why such a stringent definition? Anything else can still come back as a task on your plate. You have to context switch (not multi-task, you actually can't do that you know), and internally prioritise. Putting something on hold either requires carefully "putting something back on the shelf", or even worse, simply dropping it without care or attention.

Possibly the most unique lesson in the book is: Any attempt to optimise a process that doesn't improve the time of that processes' bottleneck is false progress. This is gold dust, and if there is nothing else in this book for you, take that. When you read the book you see things in a new light (unless you've already read The Goal). In any process - look at the bottleneck and refine that and that only, until it is no longer the bottleneck. Then find the new bottleneck. The books example is the amazing "Brent" - at the heart of every production incident, architectural planning exercise and key software project deliverable. If he gets hit by a bus, the company could literally fold. However, after several chapters of removing Brent from every coal face so that he can actually improve the overall process of the organisation comes the big reveal. Brent is not a "work centre" - he's only a person at the work centre. Like any manufacturing plant many parts of what we do are automatable - particularly in the deployment life cycle and testing. Which leads to...

Identify your work centres: Identify all the parts of your software development cycle, they are your work centres. Now find the bottleneck among those, and improve that. Remember the golden rule above - if you are not improving the bottleneck of the process, then your effort will not reap the benefits you desire.

These last two are  the current topics of my own contemplation, as we try to reduce our continuous deployment cycle down to where we know others have already reached. It finally gives me a strategy for approaching the problem that doesn't involve simply "refine all the things". It may be obvious to say "improve your worst bits first", but what you think are your "worst bits" might not actually turn out to be your bottlenecks, so improving them would be false progress.

There are lots of other excellent snippets in there (and probably some major points I glossed over, but hey, you are going to read it anyway, right?). Why ten minutes work might take several days to be acted upon for example, and these just add more and more interesting food for thought.

I'll certainly be re-reading The Phoenix project - probably any time I get stuck on how we improve next, but first up I'll be reading The Goal

Happy reading :)

Could not copy the file manifest because it was not found

This is an old chestnut that has maddened me for quite some time. It all starts with the error message

• C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Microsoft.Common.targets(4453, 5): error MSB3030: Could not copy the file "bin\Release\myapp.exe.manifest" because it was not found. 
• Project src\mysolution\myapp\myapp.csproj failed. 
• Project src\mysolution\mysolution.sln failed. 

It almost certainly happens when you are doing the following:

• Using msbuild directly, rather than Visual Studio "right click" (e.g. you are using Team City)
• Have a solution with a web application and some other application types (e.g. a console app)
• Are using the Target "Publish" to publish the website

For example, here are the TC settings that cause the problem for me:

I'm also using a solution that has this structure

• MySolution
• MyConsoleApp
• MyWebApp

MyWebApp has a publish profile setup called "PublishToDisk". If I build or publish from Visual studio, everything is fine. If I build from Team City, I get the errors above.

I've known the cause of the problem for ages. The blanket "Publish" target is being applied to every project in the solution that might be publishable. That includes my console application. My console app is not setup for publishing however, nor do I want it to be. I do, however, want it to be built during the Team City process, as the app will get copied elsewhere via the packaging process in later stages.

However the possible solutions are not so obvious. running msbuild separately on myConsoleApp.csproj and MyWebApp.csproj seems ridiculously inefficient. Making my console application a click once app just to stop a build failure seems equally silly. By far the best solution I've found is to edit your myconsoleapp.csproj file and add the following section

<PropertyGroup>
    <GenerateManifests>true</GenerateManifests>
</PropertyGroup>

You can add this just before the ItemGroup for reference includes.

This should be enough to keep to allow your application to build without a load of very awkward feeling bodges.

Incidentally - this problem is one that occurs if you try to do this publishing process with Orchard via TC, so would solve that scenario too.

Going Native is All Too Easy

We tend to fall into the same traps over time, not matter how much we try. When I was a consultant, it was very easy to correct a client who demanded technical solutions instead of listing their business objectives. It was my job to re translate that back into "what is the real requirement here".

Now I've been working for an architect at a single company for two years, I find myself falling into the traps that I used to help others avoid. In designing a new profile service a group of three architects, plus other contributors discussed the proposed structure. We went over all the technical implications, and we felt we understood what the business need was quite well thank you, including what would need to be in an MVP, and what could be deferred to later.

Four months on and I find myself in a very enlightening meeting with key business users, who have a far better understanding of what they need than I did, despite having "done my research". I find that several of the decisions I made, while valid in their way, just didn't go far enough to addressing the business need.

It is a timely reminder that the developer (no matter how well in tune with the business he believes himself to be) is ultimately far more focused on a technical challenge and "elegant solutions" than he is with what the end consumers want.

Lesson Learned: If you think you have a solution to the business problem, ask yourself "have I sat in a room for an hour with four business user who have no interest in how it is done, but only what it lets them do?"

Pros and Cons: Comparing RavenDB and FoundationDB

We've recently been evaluating options for storage for a new profiles micro-service. The original prototype for this was produced in Raven, but recently other teams within the business have been having some level of success with FoundationDB. While RavenDB is touted as a "Document Store", FoundationDB claims to be simply a "Key/Value pair store. Below is my assesment of the pros and cons of each.

RavenDB

---

Pro

• Excellent .NET client API with extensibility points, providing easy developer learning curve
• Designed as Document Store
• Provides Index/Map/Reduce
• Stores natively as JSON
• Good Read Performance
• Fits excellently into integration testing, due to in memory db option designed for testing
• Automatically generates Ids for records
• Well presented web based management studio
• Raven Server has a good console where you can see requests/response times and what indexes are used to resolve queries

Con

• Cannot test replication, sharding or authenticated access functions without purchasing licenses and licenses are needed for anything other than development (UAT would need licenses)
• Some concerns over the dependence of the RavenDB project on one key developer
• Some concerns about the robustness of the testing of the product and its unproven track record in enterprise solutions (posts like this are easy to find
• Yet another product for devops to support
• Need to understand the “eventually consistent” model well when designing solutions

FoundationDB

---

Pro

• We already use it in several other services (we have experience of it)
• Better licensing terms (by far) All features free outside production, and production licensing terms essentially means it is currently free for us to use
• Full ACID compliance
• Has both Consistency and Availability during Partitioning (assuming it isn’t a catastrophic failure)
• Built in transaction retries
• Support from FDB team is excellent
• Excellent read performance both single reads and range reads are only marginally slower
• Transaction isolation level is serializable
• Simple to scale horizontally

Con

• Designed as a Key/Value pair store rather than Document Store
• Weak .NET support (only 3rd party .NET client wrapper on top of C) with less .NET documentation/support. NET not considered first class citizen of FoundationDB
• Constraints on deployment - cannot be deployed in IIS, must be self-hosted - loss of IIS specific features such as graceful request handling, automatic app pool recycles/mem management
• Cannot run in multiple AppDomains on same process
• Like Raven, also an Alpha product, though we have less concerns about the composition of the development team
• Does not generate IDs, so a separate “ID Generation Service” would be required (or switch entire platform to GUIDs with the resulting data migrations
• No nice ‘management’ interface - you need to roll your own admin tool

To summarise the differences at high level I would say that RavenDB is great for .NET developers to rapidly write applications against, but may provide a problematic operational experience, and paying before you can test replication/"clustering" successfully is a tough ask. Conversely, FoundationDB has a really steep curve to get going with .NET, you have to live without several normally expected comforts, but does provide a compelling operational case from the ACID/clustering point of view (though, it too is an alpha, but its testing thoroughness seems far better).

As for which is "better", well, it really depends on purpose. It looks like a case of weighing up "Fast, easy development" versus "fast easy ongoing operational support". As a business we are currently leaning towards the operational ease, as products tends to spend most of in production, not development (unless you are the UK government of course).

It's great when your tools make you more effective...

Today I wrote some code, which my integration tests showed a regression in. I fixed this and merged to Git. Hub. Team City automatically build and versioned a package, which Octopus Deploy deployed to dev01. I then ran my load tests, which showed things had slowed down. I checked out the problem using New Relic, which showed traces for the slow transactions, along with our correlation token for the slow requests. I then used Kibana to find all log entries stored in Elastic Search with that correlation token, which highlighted the area causing the slow down.

I love it when a plan comes together....

A simple Orchard module to inject a diagnostics shape into every page of your Orchard site

Orchard has some great extensibility hooks. This post will show you how to very quickly use one to add a diagnostic section (like below) to the top of each page.

This uses:

• A feature in a module: to toggle on/off the ability
• A class implementing Orchards FilterProvider
• A view file to be used as a shape

First thing to do is create a new module, with a feature inside. If you need a primer, try the Orchard Walkthrough and the Hello World Module example. This article assumes that you create a module called "MyModule", and that you make a feature called "MyModule.SessionChecker"

To write output to every page we can hook into the FilterProvider class provided by the Orchard framework. Orchard.Mvc.Filters.FilterProvider is an abstract class that implements IDependency. This means that if your feature implements the class, it will automatically be wired up by Orchard at run time. All you need to do is fill in the methods of the class in your own inherited version, like below.

Code:

    [OrchardFeature("MyModule.SessionChecker")]
    public class SessionCheckerFilter : FilterProvider, IResultFilter
    {
        private readonly IWorkContextAccessor _workContextAccessor;
        private readonly IShapeFactory _shapeFactory;

        public SessionCheckerFilter(IWorkContextAccessor workContextAccessor, IShapeFactory shapeFactory)
        {
            _workContextAccessor = workContextAccessor;
            _shapeFactory = shapeFactory;
        }

        public void OnResultExecuting(ResultExecutingContext filterContext)
        {
            if (filterContext.Result as ViewResult == null) {
                return;
            }

            _workContextAccessor.GetContext(filterContext).Layout.Zones["Body"].Add(_shapeFactory.Create("diagnosticview"), ":before");
        }

        public void OnResultExecuted(ResultExecutedContext filterContext) { }
    }

The key things we are doing here are

• Using the OnResultExecuting method to ensure we hook in as the result is being formed
• Creating a new shape and returning it inside the body zone of the page 

The use of shapes is a massive topic in Orchard, and there are others out there better suited to talking about them. For this purpose it is enough to say that we use the IShapeFactory to create an arbitrary shape. The name of this shape is the name of a razor view that we need to create in our views folder in the module.

Add the file diagnosticsview.cshtml to the "views" folder in your module. The shape factory will find and use this shape. I've made my example as follows:

 <style> 
   .SessionChecker {  
     position:absolute;  
     left:0px;  
     top:0px;  
     z-index:100;  
     border:solid 1pt #AAAAAA;  
     background-color: #EEEEEE;  
     padding: 5px;  
     font-family: consolas, arial;  
     font-size: 10pt;  
   }  
   .SessionChecker b {  
     font-weight: bold;  
   }  
 </style>
 <div class="SessionChecker">  
   Current Value for Session["TestSessionManagement"]: @Session["TestSessionManagement"]  
 </div>  

And all this does is show the value of a particular session setting I'm interested in, but you could make yours much more interesting. Any loigic more complex than that shown should be done within the FilterProvider class you implemented, and be passed to the view as a model

Lastly, make sure to enable your module/feature to see the results. When you don't want them appearing any more, disable the module! Excellent for temporary diagnostic scenarios. For added bonus - only show it when the current user is an admin!

Gotcha in Orchard CMS RoutesDescriptor when using Multi-Tenancy

It's been a while since I posted, and I've been using Orchard CMS a lot among other things. I came across a rather trickysome problem today, hopefully this post will help others find the resolution quicker than I did. In orchard you can Implement IRouteProvider in any module you write. This is basically a wrapper for MVC routes, and works really well. For example, the route defined below is for custom handling in the event of errors.

---

public class ErrorHandlingRoutesProvider : IRouteProvider
    { 
        public IEnumerable<routedescriptor> GetRoutes() 
        { 
            return new[]{  
                new RouteDescriptor{ 
                    Name = "ErrorRoute",  
                    Priority = 1,                     
                    Route = new Route( 
                        "Error", 
                        new RouteValueDictionary{ 
                            {"action", "ErrorPage"}, 
                            {"controller", "ErrorHandler"}, 
                            {"area", "BG.Shared.ErrorHandling"} 
                        }, 
                        new RouteValueDictionary(),//constraints (none here) 
                        new RouteValueDictionary{ 
                            {"area", "BG.Shared.ErrorHandling"} 
                        }, 
                        new MvcRouteHandler()) 
                }; 
        }

---

This works as intended in a single site. However, when you have two tenants using the same module, this will fail with an error similar to the following:

---

System.ArgumentException: A route named 'ErrorRoute' is already in the route collection. Route names must be unique.
Parameter name: name 
   at System.Web.Routing.RouteCollection.Add(String name, RouteBase item) 
   at Orchard.Mvc.Routes.RoutePublisher.Publish(IEnumerable`1 routes) in d:\Workspaces\GitHub\src\Orchard\Mvc\Routes\RoutePublisher.cs:line 100
   at Orchard.Environment.DefaultOrchardShell.Activate() in d:\Workspaces\GitHub\src\Orchard\Environment\DefaultOrchardShell.cs:line 48
   at Orchard.Environment.DefaultOrchardHost.ActivateShell(ShellContext context) in d:\Workspaces\GitHub\src\Orchard\Environment\DefaultOrchardHost.cs:line 156
   at Orchard.Environment.DefaultOrchardHost.CreateAndActivateShells() in d:\Workspaces\GitHub\src\Orchard\Environment\DefaultOrchardHost.cs:line 135

---

The workaround is to comment out the "Name" attribute of the RouteDescriptor, as follows

public IEnumerable<RouteDescriptor> GetRoutes() 
        { 
            return new[]{  
                new RouteDescriptor{ 
                     
                    //Name = "ErrorRoute", //This doesn't work in Multi-Tenancy 
                    Priority = 1,                     
                    Route = new Route( 
                        "Error", 
                        new RouteValueDictionary{ 
                            {"action", "ErrorPage"}, 
                            {"controller", "ErrorHandler"}, 
                            {"area", "BG.Shared.ErrorHandling"} 
                        }, 
                        new RouteValueDictionary(),//constraints (none here) 
                        new RouteValueDictionary{ 
                            {"area", "BG.Shared.ErrorHandling"} 
                        }, 
                        new MvcRouteHandler()) 
                }      
            }; 
        }

I've started investigating this with the Orchard team, but the workaround doesn't really appear to have any drawbacks.

Say NO to SOPA and PIPA

Explanation of SOPA in cartoon | Wikipedia entry for SOPA

Unreadable content was found in this item - PerformancePoint 2007 to 2010 Migration

This little chestnut caused me no end of fun, and there is not a whole lot out there about it.

The Problem:

When you run the Import PerformancePoint 2007 Content wizard, using a valid account to connect to SQL server, and a valid BI Center as the target locations (which the wizard very kindly automatically identies and selects for you), you still receive the following message in each section.

"Unreadable content was found in this item".

There are two possibilities for this problem:

When it happens to all sections of the import (data sources, indicators, KPIs, Report views, score cards, dashboards):

The likely reason here is that the server needs to communicate with itself using a web service URL, and the (cursed) loopback adapter check is on. For this to be solved, the server must be able to access the target web application (e.g http://myintranet.company.com) from the local machine. This is easy to test. Open your browser from the server and see if you can. The following steps will resolve this issue:

1. Remove the loopback adapter check using the following powershell:
   
   
   

   # Disable the Loopback Check
   #This setting usually kicks out a 401 error when you try to navigate to sites that resolve to a loopback address e.g.  127.0.0.1
   New-ItemProperty HKLM:\System\CurrentControlSet\Control\Lsa -Name "DisableLoopbackCheck"  -value "1" -PropertyType dword

2. 
   

   Ensure that a host entry exists for the site, either by DNS (for already in production systems), or, if you are in testing, or don't have access to DNS, via adding the hosts entry to your hosts file at c:\windows\system32\drivers\etc\hosts, e.g.
   
   127.0.0.1 myintranet.company.com

References: http://social.technet.microsoft.com/Forums/en/sharepoint2010setup/thread/306c59e0-c74c-4f37-9df3-2b1202cef54e, http://sptwentyten.wordpress.com/2010/03/06/disable-the-loopback-check-via-powershell/

When it only happens to data sources and scorecards

My problem continued to persist past the first problem above. On further investigation of the SharePoint ULS logs, you should see messages similar to:

Failed to look up string with key "Section2TitleResource", keyfile osrvcore. edb1db92-2bd9-4dab-b772-b3b36b293e99

Unreadable content was found in this item. System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary. at System.ThrowHelper.ThrowKeyNotFoundException() at System.Collections.Generic.Dictionary`2.get_Item(TKey key) at Microsoft.SharePoint.Administration.SPScenarioContext.RetrieveDataFromSessionState(String key) at Microsoft.PerformancePoint.ImportUtility.UI.WebPages.ScenarioPageBase.Page_Init(Object sender, EventArgs e) edb1db92-2bd9-4dab-b772-b3b36b293e99

Now, this might lead you to think that something has not installed correctly, as it is complaining about resource files, but like nearly all problems in SharePoint, it comes back to permissions, and user context. The key issue in my case? I had created my own web service application pool to run the PerformancePoint service application under. The managed account for the application pool was a low privilege account (not the farm account for example).

The key thing I had missed was, that I needed to run the following PowerShell to provide my service account with appropriate object access to run the wizard successfully.

$w = Get-SPWebApplication("http://myintranet.company.com")
$w.GrantAccessToProcessIdentity("dev\svc_PPServices")

To be fair, Microsoft do list this in technet, but as with so many MS articles, they fail to tell you why, or how to recognise when you have not done this step.

references: http://technet.microsoft.com/en-us/library/ee748643.aspx

SharePoint 2010 Capacity Guidelines updated for SP1

Major changes seem to be more qualification on IOPS for large content databases, plus some more detailed understanding of the real limits of DB sizes in particular scenarios. Of particular interest is the idea that individual DBs could be up to 4TB in size (though why you would plan for one 4TB database rather than a number of more manageable ones is a different question)

http://sharepoint.microsoft.com/blog/Pages/BlogPost.aspx?pID=988

Membership & BDC import status hangs with infinite items (SharePoint 2007)

This was a bit of an odd one I came across, caused by a transaction log filling up. Take the following scenario

1. The log file for SharePoint_SSP_Search (the SSP Search database) becomes full, and cannot grow further. The database stops responding
2. The Office Search Service , then the SharePoint timer service throws an error as it cannot contact the database. This causes an error when the crawler tries to pause, and the object cache becomes corrupted (slight speculation here).
3. The SharePoint services continually attempt to communicate with the SQL Server (every second), the net result being that the server is too busy to serve RDP requests (not ideal if doing remote support!)
4. Someone “resolves” the database log size, and restarts all the SharePoint services, including IIS, the Timer Service and Office Search (or server reset)
5. Office Search and Timer resumes, persisting the corrupted object cache data to the database. This has a knock on impact on the “Membership and BDC Import”, which gets stuck in some form of infinite loop (last count, over “56,000” AD records had been imported, in an AD of less than 2,000!)
6. This fills up the transaction log at the rate of a few 100MB a minute, quickly reaching the max log file size again, causing SharePoint to stop responding once more.

The route cause of all this was that the transaction log for the search database was full. In this scenario the reasons was:

1. The log file had been capped at 5GB. This is not necessarily unreasonable, but if it does reach the limit, this triggers the issue.
2. The database is in simple recovery mode
3. The log file will grow with each SharePoint content crawl, or user profile import.
4. The database backup for SharePoint_SSP_Search appears to not be on a daily schedule (last backup was more than 10 days previous). This means that the log file has significantly more time to grow, and shrinking the log nightly has no impact, due to the fact that the database (despite being in simple mode recovery) has not been backed up, and the transaction log space has not been freed for deletion.

To resolve the problem, you can take the following steps:

1. Manually backup/shrink the search database. Note, if your database is maxed on the log file, you may need to allocate more log file space before you can succeed in a manual backup/shrink
2. Created a particular maintenance plan for this database, which backs up nightly, then shrinks the log file. (this stops the trigger for the issue from occurring under "normal" conditions.
3. Stop the troublesome “Membership and BDC Import”
4. Reset all crawled content, which clears out the corrupted data from the search database
5. Started a full user profile import and confirm success.
6. Initiated a full crawl on “Local Office SharePoint Server Sites”, and confirm success

I hope this helps someone else out. If you find a more definitive route cause or fix, please let me know.

What you can do with HTML 5

Quite a nice collaboration using jquery and HTML 5. really good showing off of new features like multiple background images and opacity...

http://disneydigitalbooks.go.com/tron/

Taken from a Waterstons press release:
----

Cookies Law and how it affects you

The laws governing privacy and the use of cookies are changing. So what's changing and how do you ensure your website is compliant with the changes?

In 2003, the Privacy and Electronic Communications Regulations required that websites using cookies for storing information, informed people of how the website uses cookies and advised them how to 'opt out' if they objected to the uses defined. The most common method to satisfy these requirements was to add detailed information to a site’s Privacy Policy and give people information on how to disable cookies within their browser.

From 26th May 2011 the Privacy and Electronic Communications Regulations will require websites that use cookies to ask users to ‘opt in’ to allow the storage of cookies on their pc, mobile device, tablet etc. This is instead of simply providing information in a website’s Privacy Policy about the use of cookies and how to disable them in common browsers.

What actions you can take

The Information Commissioner’s Office has advised that website owners should make a list of all cookies and similar technologies being used on your website and how they are used. For each one, determine how intrusive that method is, i.e. does the information track people’s habits on your site, and is the information used by third parties?

You will then need to decide which method of obtaining consent will give people the best experience on your site and fulfil your requirements. Methods include pop-up windows or requiring users to accept Terms and Conditions before they use your website.

Exceptions to the rule

This rule applies to all cookies in use on a website unless the cookie is "strictly necessary" for a service requested by a user, for example, a cookie used to maintain the contents of a Shopping Basket; however the details of these cookies and their use should still be detailed in a website’s privacy policy. An example of a cookie which would not qualify under these criteria would be those created if your website uses an analytics service.

The main message within these changes is to be transparent about how your website functions. The challenge is gaining consent from all those who visit your site, be they registered members of your services or general visitors. As website owners you will not want to alienate people from using your website and services, but instead empower them to make the correct decisions.

More Information

For more information, please refer to the Information Commissioner’s Office guide

---

For a bit of commentary on the situation, I'd highly recommend taking a look at Andrew Westgarth's "Cookies Law: Ah the Irony!

Lookups in InfoPath when you don't want the dropdown value

Sometimes you want to get a value based on a dropdown, but not actually the value in the ID column. For example, I have a list of project roles as below

RoleId	Name	Value
1	Consultant	15
2	Senior Consultant	20

Here I want a dropdown listing "Consultant" and "Senior Consultant". I need the. I then want to retrieve the corresponding value for use in a calculation. If you try this straight off, you will find that you always get the value first item from the list. This is because the context of which node you want is confused. To provide the "correct" context, you probably want an XPath query like below

../../../my:Roles/my:Role/my:RoleSaleValue[../my:RoleID = current()/../my:TaskRole]

More information can be found in this rather nice article. http://blogs.msdn.com/b/infopath/archive/2004/09/13/228881.aspx

Visual Studio Achievements

Visual Studio Achievements....I think I'm going to have to implement this one...

http://blog.whiletrue.com/2011/01/what-if-visual-studio-had-achievements/