DUnitX and my plans

I have been a big fan of using unit testing frameworks.   I use both DUnit and DUnitX for my Delphi tests.

I am right now contributing to DUnitX as I see it as the future framework of choice for Delphi Developers.

I just wanted to talk in public about some of work I have going on around DUnitX.   My goal is simple to get feedback from the Delphi community.

IDE Expert

It is really easy to setup a test project and test fixtures with DUnitX, but I thought it could be easier.

So I created a new Delphi Open Tools Expert that does the following:

• File | New | Other | Delphi Projects | DUnitX Project
• Creates a new DPR/DPROJ
• DPR Source is modeled after the DUnitX example unit tests.
• Base Project Search Path set to $(DUnitX)
• Optionally creates a Test Unit
• File | New | Other | Delphi Files | DUnitx Unit
• Creates new Delphi unit
• Adds   DUnitX.TestFramework to uses
• Creates a new class with correct attributes, you get to specify class name
• Optionally creates Setup and TearDown methods
• Optionally creates Sample Test Methods.
• Registers the TestFixture in the initialization section.

Basically it's not much, but it provides a framework to reduce your time to get to writing actual test code.    I am nearly done with this code, I wrote most of it during the Christmas Break.   Hopefully in the next week I can finish this.  It's going to take some time, as I have to build a machine with Delphi 2010 through XE5 on it to test this functionality as I only have XE and XE5 installed right now.

To see the current state of the code check out this expert branch. 
Update:  Code is now part of the master branch on the Main Project

Data Driven Test Cases

The potential to have data driven test cases is the main reason why I started looking at DUnitX.   

  [TestFixture]
  TMyTestObject = class(TObject)
  public
    [Setup]
    procedure Setup;
    [TearDown]
    procedure TearDown;
    // Sample Methods
    // Simple single Test
    [Test]
    procedure Test1;
    // Test with TestCase Atribute to supply parameters.
    [Test]
    [TestCase('TestA','1,2')]
    [TestCase('TestB','3,4')]
    procedure Test2(const AValue1 : Integer;const AValue2 : Integer);
  end;

The method Test2 above shows up as two different tests, the first time it's called with 1 and  and second time it's called with 3 and 4.

I recently made a change to underlying structure of the code. First I created a new record called TestCaseInfo followed by two new abstract attribute classes
 

 
   ///  

   ///    Internal Structure used for those implementing CustomTestCase or
   ///    CustomTestCaseSource descendants.
   ///  
   TestCaseInfo = record
 
     ///  

     ///    Name of the Test Case
     ///  
     Name : string;
 
     ///  

     ///    Values that will be passed to the method being tested.
     ///  
     Values : TValueArray;
   end;

  TestCaseInfoArray = array of TestCaseInfo;


  /// 

  ///   Base class for all Test Case Attributes.   
  /// 
  /// 
  ///   Class is abstract and should never be, used to annotate a class as a
  ///   attribute.   Instead use a descendant, that implements the GetCaseInfo
  ///   method.
  /// 
  CustomTestCaseAttribute = class abstract(TCustomAttribute)
  protected
    function GetCaseInfo : TestCaseInfo;  virtual; abstract;
  public
    property CaseInfo : TestCaseInfo read GetCaseInfo;
  end;

  /// 

  ///   Base class for all Test Case Source Attributes.   
  /// 
  /// 
  ///   
  ///     Class is abstract and should never be, used to annotate a class as a
  ///     attribute.   Instead use a descendant, that implements the
  ///     GetCaseInfoArray method.    
  ///   
  ///   
  ///     Note: If a method is annotated with a decendant of
  ///     TestCaseSourceAttribute and returns an empty TestCaseInfoArray, then
  ///     no test will be shown for the method.
  ///   
  /// 
  CustomTestCaseSourceAttribute = class abstract(TCustomAttribute)
  protected
    function GetCaseInfoArray : TestCaseInfoArray; virtual; abstract;
  public
    property CaseInfoArray : TestCaseInfoArray read GetCaseInfoArray;
  end;

With these two classes, I changed TestCaseAttribute to descend from CustomTestCaseAttribute.   
Then I changed the architecture to create a Test based on the TestCaseInfo record structure, that is obtained by either the CaseInfo or the CaseInfoArray properties of the abstract classes.

This little change provides for some really nice functionality, for example I have working sample that uses FireDAC to provide the values to my tests method

  FireDacTestCaseAttribute = class(CustomTestCaseSourceAttribute)
  protected
    FTestName : String;
    FConnectionName : String;
    FSelectStatement : String;
    function GetCaseInfoArray : TestCaseInfoArray; override;
  public
    constructor Create(const ATestName : String;const AConnectionName : String; const ASelectStatement : String);
  end;

  // Which then can be used like this:
 [FireDacTestCase('TestName','MyDBConn','select strVal, IntVal from Table');
 procedure MyTestMethod(AValue1 : String; AValue2 : Integer);

// Right now the test come out names 'TestName1', 'TestName2', 'TestName3' although that will 
// change before I commit my code to allow specifying values.  

//Most likely passing the TValuesArray with a Format call on testName like this:

 [FireDacTestCase('TestName%0:s%1:s','MyDBConn','select strVal, IntVal from Table');
 procedure MyTestMethod(AValue1 : String; AValue2 : Integer);

This attribute is not in the DUnitX.TestFramework.pas to avoiding creating dependencies on those that don't use this functionality.       This needs quite a bit of work before it's polished enough for general use, but it's what I am working on next after the expert is submitted as a pull request.  

Note: Since FireDac changed it's naming, I believe it might be XE4 or XE5 specific.

Repeat Attribute

DUnitX defines a RepeatAttribute that is currently not implemented.    I made an attempt at implementing it, and I don't like it.    If anyone has better idea, I would be happy to entertain it.

Otherwise, I have some small improvements I think I will make and will submit it again.

My implementation can be found in Pull Request #26  which won't auto merge due another change being implemented first,  but can be viewed in a working state in my RepeatAttribute branch.

Future Plans

Things I want see in a Unit Testing framework is vast, I not sure what I will start on next but here are some areas I am considering with no preference on order.  Note: this is not a road map as some may never be done (at least by me)

• Test Categories similar to NUnit as I have 10k+ of DUnit tests currently and categories might make it easier to group run on the ones I am interested in.
• VCL and Firemonkey  GUI Runner
• Understands and can filter on categories
• Quick Filter by name
• Run all or specified tests
• Simplify the DUnitX project source and make easier to select which runner is used.
• VCL
• Firemonkey
• Console
• Something similar to GUITesting.pas found in DUnit
• Load and run tests stored in BPLs.
• Data Driven Tests Enhancements
• dbExpress
• CSV Test Case Source
• XML Test Case Source
• TestCaseSource Attribute
• IOC looks up the Test Case Source Builder Interface
• Other Sources implement Test Case Builder Interface and Register it in IOC Container.
• Remote Test Framework
• Think mobile, tests are on the device the runner is on your development machine.
• Think Test Farm, multiple machines, with different configurations all running tests.
• Implement TestInOwnThreadAttribute 
• Find better way to test new functionality in DUnitX.

Thank you

And last but not least I would like to thank Vincent Parrett and his team for the set of great tools he as produced for the Delphi community.

I use the following:

• FinalBuilder - Build Automation Tool - Commercial - Well worth the price!!!!
• ContinuaCI - Build Server - Commercial - Free single for a single build server/agent.
• DUnitX - Unit Testing Framework - Open Source
• Delphi-Mocks - Mocking Framework - Open Source
• DUnit-XML - DUnit XML output in NUnit Style - Open Source

There is more Open Source he has produced all listed on GitHub.