Option for Istanbul coverage to include all source files (as opposed to only files loaded during test)

[westonpace]

Please provide us with the following information:

1. OS? Windows 7, 8 or 10. Linux (which distribution). Mac OSX (Yosemite? El Capitan?)
   Any
2. Versions. Please run ng --version. If there's nothing outputted, please run
   in a Terminal: node --version and paste the result here:

angular-cli: 1.0.0-beta.11-webpack.2
node: 6.2.2
os: linux x64

1. Repro steps. Was this an app that wasn't created using the CLI? What change did you
   do on your code? etc.

Create an application. Test some, but not all of the source files. Generated coverage will not include files that were not loaded by the tests.

1. The log given by the failure. Normally this include a stack trace and some
   more information.

N/A

1. Mention any other details that might be useful.

---

Thanks! We'll be in touch soon.

Similar to gotwarlost/istanbul#275 and karma-runner/karma-coverage#125

[jtesser]

+1
karma-coverage has an option includeAllSources which could be used.

[filipesilva]

If anyone has is willing to do a PR, or point me in the right direction to do it with webpack/karma-remap-istanbul, I'd be happy to look at it.

[jtesser]

@filipesilva what exactly are you asking for I am confused.

[westonpace]

Given that I understand webpack a little better now the includeAllSources option in karma would not work. Under the hood that is including all files that were instrumented. Webpack won't instrument code that isn't loaded so includeAllSources would not see it.

Instead I ended up using a simpler workaround. I added a single file:

src/app.module.spec.ts

which is simply (e.g. no test suites or cases):

import './app.module';

This file will cause webpack to load the entire application. The istanbul coverage will only be generated for local code (since node_modules is excluded). As far as I can tell this is a pretty ideal solution with 2 caveats:

• It is still possible for code to be missed in coverage if that code is not included in the application at all. One could potentially mitigate this with a webpack plugin that checked for unused code in ./src and reported it.
• It will cause an increase in test times depending on the size of the application.

[westonpace]

Let me know if you want me to make a pull request with this particular solution (or go ahead and implement it as it is simple enough). I wasn't sure creating a file named app.module.spec.ts would be universally accepted (some might be confused by its presence) and was hoping that someone might have a better, more invisible, idea on how to accomplish the same thing.

[filipesilva]

@westonpace I see how your solution works. It's something that's worth discussing before being implemented though...

A more direct use of your solution would be to add import './app.module'; to src/test.ts instead.

I think the biggest drawback is now that the whole app is indeed loaded on every test, whereas we only realistically want it for code coverage tests.

Not sure of what the best approach is for now, but at least we have a workaround. Let's let this issue sit open for a while, see what people say.

[seeruk]

Another option may be to modify the following line in test.ts to require everything:

.then(() => require.context('./', true, /\.spec\.ts/))
// Becomes
.then(() => require.context('./app', true, /\.ts/))

If you use lazy loaded modules importing AppModule won't include everything.

It seems like the main downside with showing coverage for everything is that you may have files that can't be covered, or don't make sense to be covered that will be included in the report, skewing your numbers a little.

[blaugold]

I'm successfully using

.then(() => require.context('./', true, /\/app\/.*\.ts$/))

in src/test.ts.
@seeruk's solution seems to trigger a weback bug in the context loader where files are loaded as directories. Putting the 'app' path part in the regex works.

[seeruk]

Aah, perhaps it needed a slash on the end of the path. I'd not actually tried that solution with angular-cli, I'm just doing something similar in the boilerplate I've been making.

[rashmi-dey]

I am not using angular-cli. Is there an option to include all source files in coverage while using karma-test-shim.js to load systemjs configuration and all test files.

[arusakov]

@filipesilva

I want to suggest separated file coverage.ts near test.ts and coverage option in .angular-cli.json.
In coverage.ts we can do all stuff related to full code coverage and developers can configure what file use for coverage via coverage option (use "coverage": "coverage.ts" - for full coverage, and "coverage": "test.ts" keeps current behaviour).

[kpaxton]

@arusakov I have implemented something similar to what you are suggesting in my own application. The downfall to this is that I then have two test start points that need to stay in sync when things might be changed.
I implemented a coverage.ts in the same folder as test.ts and updated the karma-conf.js to actually change the files array and preprocessors object to reflect the new start point if config.angularCli.codeCoverage is true.
It works pretty well, but having a flag available to test.ts to allow for switching the context based on that flag I think would be slightly more ideal than having two separate files.