Tuesday 23 September 2014

Lamda expressions in unit testing are hard

So, it turns out that Lamda expressions are not great for unit testing. A recent example of code that I wanted to unit test:

 public async Task RegisterIndexAsync<T>(IFoundocIndex<T> index, CancellationToken cancellationToken)  
     {  
       //.....  
       await _fdbStorageProvider.ReadWriteAsync(async transaction =>  
       {  
         var indexDefinitionState = await _indexProvider.GetIndexDefinitionStateFromStore(transaction, index).ConfigureAwait(false);  
         if (!indexDefinitionState.Exists || indexDefinitionState.Changed)  
         {  
           await _indexProvider.PersistIndexToStore(transaction, index).ConfigureAwait(false);  
           var documentCount = await _documentProvider.Count<T>(transaction).ConfigureAwait(false);  
           if (documentCount > 0)  
           {  
             Trace.WriteLine(documentCount + " items found in collection");  
             rebuildIndex = true;  
           }  
         }  
       }).ConfigureAwait(false);  
       if (rebuildIndex)  
       {  
         Trace.WriteLine("Rebuilding Index: " + index.Name);  
         await RebuildIndexAsync(index, cancellationToken).ConfigureAwait(false);  
       }  
       Trace.WriteLine(String.Format("Not Rebuilding Index: {0}.", index.Name));  
     }  
     public async Task RebuildIndexAsync<T>(IFoundocIndex<T> index, CancellationToken cancellationToken)  
     {  
       using (var queue = new BlockingCollection<IEnumerable<T>>(_settings.MaxBatchesInIndexQueue))  
       {  
         _batchEntityProvider.GetBatches(queue, _settings.MaxIndexBatchSize).ConfigureAwait(false);//deliberately not awaiting this  
         await _batchConsumer.Consume(queue, cancellationToken, async batch => await ConsumeWorkQueue(batch));  
       }  
     }  

Now of course, I'm unit testing an implementation call to RegisterIndexAsync<T>(IFoundocIndex<T> index, CancellationToken cancellationToken) but I also want to verify that in this test, that my index was not rebuilt. Normally you could do this by mocking (for example, using the amazing Moq and verifying the number of calls to_batchEntityProvider.GetBatches but here there is a complication.

In this example you would need to use a Setup operation on _fdbStorageProvider.ReadWriteAsync that would supply the entire of the lamda expression as its setup. Essentially - you would need to know and express the code for this function in your unit test setup. Your unit test becomes essentially "ensure that what the code does is what the code does" - and this is not right.

It is also extremely hard to do due to the way lamda expressions compile - the same resulting code will compile to a different object - so they are never going to the same object in your Moq setup call.

Looking more deeply into this example you could validly say that I shouldn't care about verifying whether an internal operation is called. All I should be worried about are external results right?

Quite possibly true in this case - if I cannot observe the impact through external interfaces its probably not worth knowing right? Except that I need to know if this method is rebuilding an index unnecessarily. If it does, there will be no observable difference - the index would be the same before and after, the only difference would be the time taken on larger indexes - something you can't identify in a unit test, and using time taken as a part of a test is a lousy idea anyway.

For this specific test, I think it is time to head back to integration tests, which leads to the future problem of - how do you do the equivalent of "Verify" when you are not mocking your classes. That's for another time.


No comments:

Post a Comment