azure-pipelines-tasks: Exception : Index was out of range. Must be non-negative and less than the size of the collection. on VSTest\2.0.17\TestSelector\TestSelector.exe UpdateTestResults

Hi,

Today I was playing with the ‘Run only impacted tests’ feature of the ‘Run Tests using Visual Studio task’. My test run succeeds, but I get the following error when VSTest\2.0.17\TestSelector\TestSelector.exe UpdateTestResults is executed (see below).

2017-03-07T15:55:38.0425386Z Total tests: 1206. Passed: 1200. Failed: 0. Skipped: 6.
2017-03-07T15:55:38.0425386Z Test Run Successful.
2017-03-07T15:55:38.0425386Z Test execution time: 11.8204 Minutes
2017-03-07T15:55:38.9516587Z Results File: C:\tfsagent\_work\19\s\TestResults\tfsonlinebuild_VTFS-DBS02 2017-03-07 15_44_13.trx
2017-03-07T15:55:39.1116416Z ##[debug]rc:0
2017-03-07T15:55:39.1116416Z ##[debug]success:true
2017-03-07T15:55:39.1116416Z ##[debug]rm -rf C:\Users\TFSONL~1\AppData\Local\Temp\bbaf38b0-034c-11e7-a552-4390675c08db.runsettings
2017-03-07T15:55:39.1116416Z ##[debug]removing file
2017-03-07T15:55:39.1116416Z ##[debug]defaultRoot: 'C:\tfsagent\_work\19\s\TestResults'
2017-03-07T15:55:39.1116416Z ##[debug]findOptions.followSpecifiedSymbolicLink: 'true'
2017-03-07T15:55:39.1195144Z ##[debug]findOptions.followSymbolicLinks: 'true'
2017-03-07T15:55:39.1195144Z ##[debug]matchOptions.debug: 'false'
2017-03-07T15:55:39.1195144Z ##[debug]matchOptions.nobrace: 'true'
2017-03-07T15:55:39.1195144Z ##[debug]matchOptions.noglobstar: 'false'
2017-03-07T15:55:39.1195144Z ##[debug]matchOptions.dot: 'true'
2017-03-07T15:55:39.1195144Z ##[debug]matchOptions.noext: 'false'
2017-03-07T15:55:39.1195144Z ##[debug]matchOptions.nocase: 'true'
2017-03-07T15:55:39.1216417Z ##[debug]matchOptions.nonull: 'false'
2017-03-07T15:55:39.1216417Z ##[debug]matchOptions.matchBase: 'false'
2017-03-07T15:55:39.1216417Z ##[debug]matchOptions.nocomment: 'false'
2017-03-07T15:55:39.1216417Z ##[debug]matchOptions.nonegate: 'false'
2017-03-07T15:55:39.1216417Z ##[debug]matchOptions.flipNegate: 'false'
2017-03-07T15:55:39.1288242Z ##[debug]pattern: 'C:\tfsagent\_work\19\s\TestResults\*.trx'
2017-03-07T15:55:39.1288242Z ##[debug]findPath: 'C:\tfsagent\_work\19\s\TestResults'
2017-03-07T15:55:39.1288242Z ##[debug]statOnly: 'false'
2017-03-07T15:55:39.1288242Z ##[debug]findPath: 'C:\tfsagent\_work\19\s\TestResults'
2017-03-07T15:55:39.1288242Z ##[debug]findOptions.followSpecifiedSymbolicLink: 'true'
2017-03-07T15:55:39.1288242Z ##[debug]findOptions.followSymbolicLinks: 'true'
2017-03-07T15:55:39.1288242Z ##[debug]  C:\tfsagent\_work\19\s\TestResults (directory)
2017-03-07T15:55:39.1316419Z ##[debug]  C:\tfsagent\_work\19\s\TestResults\e8ca5e74-0e5e-4b9a-9669-ae42f540193d (directory)
2017-03-07T15:55:39.1316419Z ##[debug]  C:\tfsagent\_work\19\s\TestResults\e8ca5e74-0e5e-4b9a-9669-ae42f540193d\tfsonlinebuild_VTFS-DBS02 2017-03-07 15_43_51.coverage (file)
2017-03-07T15:55:39.1316419Z ##[debug]  C:\tfsagent\_work\19\s\TestResults\f82cb885-fd7f-461c-a1ec-97ade1afe849 (directory)
2017-03-07T15:55:39.1316419Z ##[debug]  C:\tfsagent\_work\19\s\TestResults\tfsonlinebuild_VTFS-DBS02 2017-03-07 15_44_13 (directory)
2017-03-07T15:55:39.1316419Z ##[debug]  C:\tfsagent\_work\19\s\TestResults\tfsonlinebuild_VTFS-DBS02 2017-03-07 15_44_13\In (directory)
2017-03-07T15:55:39.1384786Z ##[debug]  C:\tfsagent\_work\19\s\TestResults\tfsonlinebuild_VTFS-DBS02 2017-03-07 15_44_13\In\VTFS-DBS02 (directory)
2017-03-07T15:55:39.1384786Z ##[debug]  C:\tfsagent\_work\19\s\TestResults\tfsonlinebuild_VTFS-DBS02 2017-03-07 15_44_13\In\VTFS-DBS02\tfsonlinebuild_VTFS-DBS02 2017-03-07 15_43_51.coverage (file)
2017-03-07T15:55:39.1384786Z ##[debug]  C:\tfsagent\_work\19\s\TestResults\tfsonlinebuild_VTFS-DBS02 2017-03-07 15_44_13.trx (file)
2017-03-07T15:55:39.1384786Z ##[debug]9 results
2017-03-07T15:55:39.1384786Z ##[debug]found 9 paths
2017-03-07T15:55:39.1384786Z ##[debug]applying include pattern
2017-03-07T15:55:39.1384786Z ##[debug]1 matches
2017-03-07T15:55:39.1384786Z ##[debug]1 final results
2017-03-07T15:55:39.1384786Z ##[debug]C:\tfsagent\tasks\VSTest\2.0.17\TestSelector\TestSelector.exe arg: UpdateTestResults
2017-03-07T15:55:39.1384786Z ##[debug]Build.BuildId=22122
2017-03-07T15:55:39.1416422Z ##[debug]System.TeamFoundationCollectionUri=https://xxx.visualstudio.com/
2017-03-07T15:55:39.1416422Z ##[debug]System.TeamProject=MFS
2017-03-07T15:55:39.1416422Z ##[debug]SystemVssConnection auth param AccessToken = ********
2017-03-07T15:55:39.1478419Z ##[debug]exec tool: C:\tfsagent\tasks\VSTest\2.0.17\TestSelector\TestSelector.exe
2017-03-07T15:55:39.1478419Z ##[debug]Arguments:
2017-03-07T15:55:39.1478419Z ##[debug]   UpdateTestResults
2017-03-07T15:55:39.1478419Z [command]C:\tfsagent\tasks\VSTest\2.0.17\TestSelector\TestSelector.exe UpdateTestResults
2017-03-07T15:55:39.1478419Z ##[debug]Deleting the response file C:\Users\TFSONL~1\AppData\Local\Temp\d847ee40-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:39.1478419Z ##[debug]rm -rf C:\Users\TFSONL~1\AppData\Local\Temp\d847ee40-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:39.1478419Z ##[debug]Deleting the discovered tests file C:\Users\TFSONL~1\AppData\Local\Temp\bd966f40-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:39.1478419Z ##[debug]rm -rf C:\Users\TFSONL~1\AppData\Local\Temp\bd966f40-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:39.1478419Z ##[debug]removing file
2017-03-07T15:55:39.1478419Z ##[debug]Deleting the baseline build id file C:\Users\TFSONL~1\AppData\Local\Temp\bb56b871-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:39.1516430Z ##[debug]rm -rf C:\Users\TFSONL~1\AppData\Local\Temp\bb56b871-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:39.1516430Z ##[debug]removing file
2017-03-07T15:55:41.4382356Z PERF: GetTestCaseResults: took 1738.5298 ms
2017-03-07T15:55:41.6221835Z PERF: ReadResultsFromFile: took 173.4023 ms
2017-03-07T15:55:42.2365842Z PERF: UpdateResultsAndSignatures: took 623.0815 ms
2017-03-07T15:55:42.2614184Z Exception : Index was out of range. Must be non-negative and less than the size of the collection.
2017-03-07T15:55:42.2614184Z Parameter name: index
2017-03-07T15:55:42.2614184Z  Stack :    at System.ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument argument, ExceptionResource resource)
2017-03-07T15:55:42.2614184Z    at System.Collections.Generic.List`1.get_Item(Int32 index)
2017-03-07T15:55:42.2614184Z    at Microsoft.TeamFoundation.DistributedTask.Task.TestResults.TestRunPublisher.UpdateResults(TestCaseResultData[] testResults, List`1 tcmTestCaseResults, TestHttpClientBase testImpactClient, Int32 runId, Int32 definitionRunId, DefinitionType definitionType) in D:\v2.0\A1\_work\4\s\DistributedTask\Agent\Task.TestResults\TestRunPublisher.cs:line 152
2017-03-07T15:55:42.2614184Z    at Microsoft.TestFoundation.TestImpact.TestSelector.CommandUpdateTestResults.Run() in D:\v2.0\A1\_work\4\s\Ta\Tasks\TestImpact\TestSelector\Commands\Command\CommandUpdateTestResults.cs:line 133
2017-03-07T15:55:42.2614184Z    at Microsoft.TestFoundation.TestImpact.TestSelector.Program.RunCommand(String commandName, String[] args) in D:\v2.0\A1\_work\4\s\Ta\Tasks\TestImpact\TestSelector\Program.cs:line 41
2017-03-07T15:55:42.2964179Z ##[debug]rc:4294967295
2017-03-07T15:55:42.2964179Z ##[debug]success:false
2017-03-07T15:55:42.2964179Z ##[debug]Test Run Updation failed!
2017-03-07T15:55:42.2964179Z ##[debug]task result: Failed
2017-03-07T15:55:42.3064182Z ##[error]Vstest exited with return code: 1.
2017-03-07T15:55:42.3364192Z ##[debug]Deleting the response file C:\Users\TFSONL~1\AppData\Local\Temp\d847ee40-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:42.3364192Z ##[debug]rm -rf C:\Users\TFSONL~1\AppData\Local\Temp\d847ee40-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:42.3364192Z ##[debug]Deleting the discovered tests file C:\Users\TFSONL~1\AppData\Local\Temp\bd966f40-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:42.3364192Z ##[debug]rm -rf C:\Users\TFSONL~1\AppData\Local\Temp\bd966f40-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:42.3364192Z ##[debug]Deleting the baseline build id file C:\Users\TFSONL~1\AppData\Local\Temp\bb56b871-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:42.3364192Z ##[debug]rm -rf C:\Users\TFSONL~1\AppData\Local\Temp\bb56b871-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:42.3364192Z ##[debug]Deleting the run id fileC:\Users\TFSONL~1\AppData\Local\Temp\bb56b870-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:42.3364192Z ##[debug]rm -rf C:\Users\TFSONL~1\AppData\Local\Temp\bb56b870-034c-11e7-a552-4390675c08db.txt
2017-03-07T15:55:42.3364192Z ##[debug]removing file

Am I missing something?

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 37 (19 by maintainers)

Most upvoted comments

@wouterroos I got a repro and I am working on a fix. Will keep you posted

We are experiencing what seems to be the same issue: checking the “Run only impacted tests” box causes our build to fail with lots of “The test source file … provided was not found” errors. We are also using theories in xUnit, although we are on v2.3.0 beta 1. Please let me know if there is anything we can do to help with testing or a fix. Since this issue is closed, is there somewhere we can track progress for always running theories or for handling them fully?

In the meantime, I just uncheck that box, and it works fine. It would be very nice to skip some of our tests – it takes about half an hour for around 3000 tests to run.