Monday, May 20, 2013

Providing mocks through data providers

Over the weekend I played a bit with TestNG data providers for the first time. This was in a scala program, but doesn't really have anything to do with scala per se. I had something like this in my test class. This is an example, it may not actually run, but it's representative of what I'd written. Key here is that the mock objects were being put together and set up in the data provider. The mocking framework here is Mockito, but I imagine this type of problem could crop up with other frameworks too.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
class CarTest extends ShouldMatchers {

  var trackMock1: Track = _
  var positionMock1: Position = _
  var trackMock2: Track = _
  var positionMock2: Position = _

  @BeforeMethod
  def setupMocks() {
    trackMock1 = mock(classOf[Track], "track1")
    positionMock1 = mock(classOf[Position], "pos1")
    trackMock2 = mock(classOf[Track], "track2")
    positionMock2 = mock(classOf[Position], "pos2")
  }

  @DataProvider(name = "positionsProvider")
  def positionsProvider() = {
    def makeTestPosition(position: Position, track: Track) = {
      when(position.track).thenReturn(track)
      Array[Object](position, track)
    }
    Array(makeTestPosition(positionMock1, trackMock1),
          makeTestPosition(positionMock2, trackMock2))
  }

  @Test(dataProvider = "positionsProvider")
  def carPositionHasRightTrack(position: Position, track: Track) {
    val underTest = Car(position)
    car.track should be (track)
  }
}

This doesn't work, with a failure message saying that Track track2 does not match Track track2. The debugger showed me the objects being compared indeed had different identifiers, which led me to conclude that the test was being run as two independent method executions, while the data provider was executed only once. The mocks changed values on each execution of the test method, while the data provider kept the mock values from the first execution. The fix isn't entirely straightforward, and required a combination of changes:
  1. Create mocks whose methods you want to provide specific behavior within the data provider.
  2. Don't re-create the other mocks. Just reset them.
My tests involved many interacting objects, and several test cases to check those interactions. I didn't like duplicating the mock setup code from scratch for each test. However, reset on all mocks (including the Position mocks) would cause loss of behavior between test cases. So the creation of the Position mocks had to move into the data provider. This version of the test ran correctly:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
class CarTest extends ShouldMatchers {

  val trackMock1: Track = mock(classOf[Track], "track1")
  val trackMock2: Track = mock(classOf[Track], "track2")

  @BeforeMethod
  def setupMocks() {
    reset(trackMock1)
    reset(trackMock2)
  }

  @DataProvider(name = "positionsProvider")
  def positionsProvider() = {
    def makeTestPosition(track: Track) = {
      val positionMock = mock(classOf[Position])
      when(positionMock.track).thenReturn(track)
      Array[Object](positionMock, track)
    }
    Array(makeTestPosition(trackMock1),
          makeTestPosition(trackMock2))
  }

  @Test(dataProvider = "positionsProvider")
  def carPositionHasRightTrack(position: Position, track: Track) {
    val underTest = Car(position)
    car.track should be (track)
  }
}

Interestingly, Mockito developers seem to have been very concerned about the potential for reset to lead to poor coding practices, and had to be convinced that there were legitimate uses for it. While my code doesn't fall into the situations they had foreseen, calls to reset are confined to the test setup.