Add flag which makes pytest to return 0 exit code if no test functions were executed.

Followup from https://github.com/pytest-dev/pytest/issues/812#issuecomment-517869154 and I’m aware about discussion in https://github.com/pytest-dev/pytest/issues/500 .

• a detailed description of the bug or suggestion

Title. With this flag added calling pytest --the-flag will return 0 exit code if no test function were found.

There are 2 cases to consider:

• there is no test functions in directory,
• all tests were deselected.

I only care about the 1st scenario because my use case it to configure CI with tox where testing is one of build steps and I want to have that step in place even if there are no tests yet. Without this flag I would have to do something (e.g. add - pytest in tox.ini) and redo that after some tests were added or perform additional bash scripting which would analyze the test directory and returned code and possibly change the return code accordingly.

Also given that people from other testing tools e.g. unittest are used to scenario “if there is no tests there is no reason to fail” so this flag will give everyone an option about how pytest should behave.

The 2nd mentioned scenario can also have such option but on one side you want to treat “all tests were deselected” as a way to prevent errors. On the other side, why don’t give users an option to configure pytest to their needs? Both, being arbitrary or open for options, have good reasons so it’s hard for me to decide on that one.

Furthermore there may come a question “if you can configure this return code then why shouldn’t it be possible to configure all of them?”. I only want to mention this because it is an extention to this issue and probably requires a separate feature request.

• pytest and operating system versions

pytest=5.0.1 OS=Arch Linux (Linux-5.2.2-arch1-1-ARCH-x86_64-with-glibc2.2.5)