In Memory Testing with NHibernate 3.x & SQL Server 4.0 Compact

Recently I started to look for ways to make testing quicker on a project I’m working on. There are a large number of integration tests which take quite awhile to run. So I decided to start by switching to an in-memory db when the tests are running. I ended up picking SQL Server 4.0 Compact because I haven’t had a lot of luck with SQLite in the past but that decision lead to it’s own.. fun which I’ve decided to detail here with the steps I took:

  • Download SQL Server Compact from http://www.microsoft.com/sqlserver/en/us/editions/compact.aspx and install it.
  • Created a new project and used Nuget to reference NHibernate 3.2 and SQLServerCompact. One thing I did have to do was select the System.Data.SqlServerCe in the References and change the Copy Local to true for NHibernate to have access to it.
  • Now is where things started to get fun. I originally created a SQL Server Compact Db using SQL Server Management Studio. I quickly found out though that this actually creates a 3.5 DB which is not compatible with 4.0 so I had to download the SQL Server Compact Toolbox from http://sqlcetoolbox.codeplex.com/ to create my database. This tool is.. sparse.. but you can at least create your table structure with it.

Now I needed to decide how to handle inserting my test data. We’ve been using NDbUnit a lot on my current project and I am honestly quite tired of the xml it requires so I wanted a change. I decided that for this little experiment to just try using a database file pre setup with the data I wanted and reset the db state between tests by rolling back the transaction.

  • Created a SQL script which would insert my tables & data. It’s at this point that I ran into a bit of a problem with the SQL CE Toolbox which was that I could only get a single statement a time to execute. If my script contained multiple statements I would get a parsing error. After a quick trip to stackoverflow I ended up downloading the Visual Studio Add-in version of the SQL CE Toolbox and modified my script so that each line was separated with a GO statement and things started working.

Next up is configuring NHibernate. Now I’ve taken a look at the new NH 3.x ‘Loquacious’ configuration and while I like it I’m not a big fan of xml based configuration OR ConfORM configuration. Instead I like to stick with Fluent NHibernate even though at the time of this writing it only supports 3.1. If you’re not using FNH then you’ll have to take the reigns at this point. Also I should note I’m using NUnit but the same concepts should be available in any test framework.

First up we need to create a session factory ONCE for all of our tests.

[SetUpFixture]public class IntegrationTestSetup{ public static ISessionFactory SessionFactory { get; set; } [SetUp] public void OnceBeforeAllTestsInNamespace() { SessionFactory = Fluently.Configure() .Database(MsSqlCeConfiguration.Standard.ShowSql() .ConnectionString(c => c.Is(@"data source=Tests\testdata\customers.sdf"))) .Mappings(m => m.FluentMappings.AddFromAssemblyOf<Customer>()) .BuildSessionFactory(); } [TearDown] public void OnceAfterAllTestsInNamespace() { SessionFactory.Close(); }}

Next up I like to create a base class for all my tests so they can have quick access to things.

[TestFixture]public abstract class IntegrationTestBase{ protected ISession Session { get; private set; } [SetUp] public void OnceBeforeAllTests() { Session = IntegrationTestSetup.SessionFactory.OpenSession(); Session.BeginTransaction(); } [TearDown] public void OnceAfterAllTests() { Session.Transaction.Rollback(); Session.Close(); }}
And just for completeness sake.. here is a test.
[TestFixture]public class CustomerTests : IntegrationTestBase{ [Test] public void Should_be_able_to_load_basic_customer() { var customer = Session.Load<Customer>(1); Assert.IsNotNull(customer); }}

Whew! That made me reach for a cold juice. If you're like me who mostly drinks natural juice, whether fruit or veggie, visit these pages: breville juicer reviews and bestjuicerreviews.xyz.


Presenting? Make sure you remember to bring your personality…

So today at my current contract was a 1.5 hour long “All Hands” meeting.  Also known as “What have we been up to this last year and where are we going?”.  As a contractor I find I have quite a few different thoughts on these kinds of events.  The first thought is “This is a waste…” which from a certain point of view it is.  Quite often the things being talked about are going to come to full fruition over many years.  Since a big part of the reason I contract is to see how different companies and teams do things I don’t intend to stay too many years at one place (I also wouldn’t want to give the tax man the mistaken belief that I’m an employee) so it’s unlikely I’ll see the outcomes. 

At the same time though I find quite a bit of value in attending and listening.  The first from a purely professional point of view.  Is this company somewhere that I am going to want to come back to in the future?  If I don’t like where things are headed (really? all future development is going to be waterfall?) then I know to be careful the next time I get offered a contract at that company.  The other big value though is when they have multiple people presenting.  As an occasional presenter myself I find one of the best ways to learn is to watch others do it.  So what did I learn this time?

Rule #1 of presenting should be “Bring Your Personality”

Obviously todays presentation was filled with people who followed and people who broke that rule.  A cookie for you if you can guess which ones got the laughs and applause (and were paid attention to). 

It’s really simple fellow presenters.. if people wanted to gain their information from a cold, boring source they would just read the news release.  They are there because they care about what you’re talking about and it helps so much to reach them if you let them know that you care too.  Show them that you excited about what what you’re talking about and how it’s going to improve their lives in some way.  By letting your personal side show (even just a little bit) you’ll get a much better response and have a much bigger impact on people.


WCF Host does not accept connections

We recently ran into a fun little problem at work after adding a new operation to our WCF service.  The first step for us after adding a service is to make that service call available to our client by regenerating the proxies it uses.  We do this by running our service and using SvcUtil against it.  Today though we ran into an interesting little problem where SvcUtil was telling us that the machine was not accepting connections.  We checked back and our service (self hosted I should point out) was still running fine.  

So at this point we started the debugging process.  First step was to check all our log points which turned up nothing.  The SvcUtil error message mentions checking for an inner exception but it doesn't mention how to go about doing that.  The fact our service wasn't throwing an error made this even more difficult to debug.  So now we started the process of elimination.  Taking a look at the code the one thing I did notice was that the only "different" thing the new code did was return an enum as part of the [DataContract].  I looked at the enum and everything looked fine (yes that was foreshadowing..) but I still thought it was the most likely candidate so I commented it out and tried again.  Everything worked fine.  At this point I went and looked at the enum declaration again.. it actually looked like this..

    [DataContract]
    public enum AuditType
    {
        [DataMember]
        Insert = 1,
        [DataMember]
        Update = 2,
        [DataMember]
        Delete = 4
    }

Now if you've done any WCF work hopefully the [DataMember] will pop out at you immediately since that is typically used to identify properties.  A quick search on Google identified [EnumMember] as the proper attribute and we were off to the races.

So first off I'm hoping someone somewhere finds this and saves themselves a little bit of hunting.. but most importantly I'm hoping someone can help me identify a better way to have figured this out.  The service compiled and ran fine.  I could not find a single error anywhere but this just seems very wrong.  So to you I ask.. what would you have done differently to have found this quicker?

* I wonder if tracing would have helped?

NHibernate, manual flushing, performance and you!

Recently on my current project we decided to sit down and take a look at performance problems that we were having.  The main cause for concern was actually our most common and important operation since it would take anywhere from 20 to 40 minutes to complete.  

The first obvious step was to run a profiler against our code to see where we were actually spending our time and for this we choose DotTrace by JetBrains.  I really liked the ANTS Profiler time line view but for reasons that I can't recall right now we decided DotTrace was a better fit for our needs (I think it had to do with ease of comparing different runs).  So with DotTrace in hand we quickly found out that most of the time in our application was actually being spent in NHibernate (specifically in analyzing entity relationships).  This came as quite a shock to me since I know NHibernate is widely used and I had never heard of anything even approaching what we were seeing.

To cut a long story short while I did look into the NHibernate code it was eventually a bit of experimentation in our own code base that helped resolve the issue.  You see in our code it's very important that we track every step in our main process and to do this tracking we were using the actual entity Db Ids and the only way we knew to do so was to tell NHibernate to save the entity and then force a flush.  By removing the flush I was able to cut a 40 minute process down to 4 minutes!  The theory I ended up with is that each time we called Flush we were creating more and more stuff that NHibernate had to parse through each time Flush was called.  By only calling it once at the end it started with a nice clean slate.

So with all that said my advice to those of you who want to manually call Flush is.. DON'T! Not unless you really really need to and even then you should seriously consider other options/designs.

Oh and as for the whole Id thing we ended up using Session.Persist(entity) to get the Ids we need.  There is a bit of confusion around this method and if it's guaranteed to provide an Id but so far with our mappings we haven't had an issue.

Things I’ve learnt lead to badness…

    Some of these I’ve done.. some I’ve just seen.. but they are all now things I watch for…

  1. Never talking to someone who is going to physically use your solution…
  2. Not having acceptance tests…
  3. Showing up for your new job and being handed a few binders worth of the design that someone who has already quit did…
  4. Working at the same job for so long that it makes you think about becoming an electrician…
  5. Not practicing TDD…
  6. Having only half a team that understands DDD but trying to do it anyway…
  7. Not listening to your gut for a hiring decision…
  8. Writing integration tests to do the job of acceptance tests...
  9. 3 BAs for 10 developers..
  10. 2 QA people for 10 developers…
  11. Not understanding your audience…
  12. Listening too much to others…
  13. Too much reading.. not enough doing…
  14. Thinking that you know everything…
  15. Believing that you know nothing…

I could keep going but this seems like a good enough place to stop for my upcoming rant…


Rounding up to the nearest 0.05

Recently I ran into an interesting problem on a small project I was working on.  The problem was on the surface fairly simple.  Take a currency amount and round it up to the nearest 0.05.  While the solution is not earth shattering it took long enough to figure out that I figured it was worth recording for my future self. 

The first part of the problem is to round to 0.05 which is accomplished by doing this (thanks Google..):

decimal initialValue = 0.56;decimal value = Math.Round(initialValue / 5, 2) * 5;

But the problem is this code will actually round to the nearest 0.05 as opposed to only rounding up.  So 0.62 will round to 0.60 where as our requirement is to round to 0.65.

But that part of the problem is actually fairly easy to solve..

 

if((value - initialValue) < 0){ value += 0.05m;}
 

And there you have it.  We can now round up to the nearest 0.05.


Edmonton Code Camp Session

So this weekend I had the chance to give a presentation at the Edmonton Code Camp.  I’ve done this every year for the last few years now (and I’ve done a few presentations elsewhere) and I have to say that this by far was my…. worst presentation ever. 

So what went wrong?  Well the primary problem I think was based on the fact that my original presentation idea was going to be a fair bit more advanced then the talk I ended up giving.  Originally my idea was to talk mainly about the differences between Unit and Integration tests.  While there are some fundamental differences between the two I honestly don’t know that I could spend over an hour talking about them.  As I worked on my presentation though I realized that there was probably a lot of value in talking about some more basic testing stuff first.

So this is where my mistake happened.  At this point I should have probably thrown out all of the advanced stuff I wanted to talk about and started fresh.  Instead I tried to walk this line of keeping in line with my original idea while trying to talk about the basics of testing first.

BAD IDEA…

I found the talk quite dysfunctional because it tried to bounce between intro level and advanced level which never works well.  So with that said here are a few things I take away from this..

1. Intro and Advanced level talks have totally different audiences.  Pick one.

2. If your original idea is wrong.. that’s fine but throw it out and start fresh.  You may bring some stuff forwards but it’s good to start with a fresh mind.

3. Semi unrelated but I really need to get started on my blog series on testing I wanted to write.  Good news is I mentioned it while talking to a few people after my talk so now I have no choice.

Living and Learning…


Expensive WCF Operations cause unresponsive service

Recently I ran into a situation where one of our WCF Operations was taking a long time to complete (in the range of a few minutes).  The problem was that while this was going on the server stopped responding to all other responses. 

After a bit of time with Google I was able to find a post that seemed to explain the problem we were having WCF Threading Internals (Updated)

At the end of the day the solution (which took a bit more work to find) turned out to be fairly simple.  It involves making your Operation asynchronous via the AsyncPattern contract parameter (and a bit of other stuff).

So lesson of the day is.. if your WCF Operation is going to take a long time.. Asynch it.


Soft Skills Book Series – My Job Went To India Update

I’m happy to announce that the “My Job Went To India…” book I talked about in an earlier post is being re-released.  This time the title has been changed to The Passionate Programmer: Creating a Remarkable Career in Software Development

I’m quite happy to see the re-release and name change.  I think the earlier name cast this book in a way that might have limited its audience.  This is quite unfortunate when you consider how valuable this book is for every developer to read. 

I’ve got the earlier version but I’m definitely going to pick this one up as well since I found the first one so valuable.


Still Alive…

Well the last month has been a bit hectic around here.  Between moving and being sick twice I’m just getting to the point where I’ll have some time to type out some of the random thoughts and maybe a few more book recommendations….


Contact me here: me@shane.jscconsulting.ca