Why Is 100% Test Coverage Easier To Achieve?
April 17th, 2011
Some say that “it’s little valuable to achieve test coverage over 80%” (or 60% in some cases). From the perspective of quality assurance, maybe it’s true. However, from a lean perspective, 100% test coverage could be even easier to achieve than 80% (and much easier than 60%). Sounds weird? I’ll explain.
The most common argument against “100% coverage” is “Do we really need test trivial methods such as getter/setter?” But it’s a bad question. What really matters is not “do we really need test them”, but “do we really need write them”. I saw programmers wrote (or even generated) getter/setters without test just because doing so is cheap. But these small pieces of code are waste: not only because you need maintain them, but also because they (potentially) break design principles such as Law of Demeter . Creating trivial methods without test-driven is over production. It’s MUDA .
And the tolerance of code-without-test causes MURA (meaning irregular, uneven or inconsistent). Lower the bar of test coverage to 80% (or even 60%), and you’ll end up to solve code-without-test in a bigger batch. It requires more consideration and effort every time you doing it, and this effort is not split into every story (or even better, every commit) evenly. You’ll need a concentrated hour (or worse, half day) to write tests for code committed a few days ago.
And as we all know, MUDA and MURA causes MURI (overburden, unreasonableness or absurdity). When it comes to the last day before your release and suddenly test coverage drops under the bar, how possible can you get a concentrated hour to fix it? Isn’t it annoying to be requested writing some test to fix broken continuous integration when you are ready to pack your laptop and go home at 6PM? Are you going to do some serious code review before typing in yet another non-sense test case and commit it?
Allowing MUDA to accumulate causes MURA. MURA plus delivery pressure causes MURI. MURI stops you from improving code quality and causes more MUDA. The lower the bar, the worse the case. You can break the chain by eliminating every piece of code-without-test continuously. Test-driven every piece of code.
Bottom line: although it might be little valuable to tighter test coverage restriction from 80% to 100% from a quality assurance point of view, doing so makes your life easier (instead of harder). So, why not doing it?
Run Multiple Marathon Testcases In One Application
September 26th, 2010
# This goes to Fixtures/default.py
functional_test_dir = os.path.split(__file__)[0] + "/.."
# And this is TestCases/all_tests.py
#{{{ Marathon Fixture
from default import *
#}}} Marathon Fixture
import sys
import os
test_cases_dir = functional_test_dir + "/TestCases"
def load_test_cases(directory):
test_cases_directory = test_cases_dir + "/" + directory
sys.path.append(test_cases_directory)
dir_list = os.listdir(test_cases_directory)
test_case_files = filter(lambda fname:(fname.endswith(".py")), dir_list)
return map(lambda fname:fname.split('.')[0], test_case_files)
test_cases = load_test_cases("open_tests")
test_cases += load_test_cases("view_tests")
for test_case in test_cases:
exec "import %s" % test_case
def test():
for test_case in test_cases:
exec "%s.test()" % test_case
测试驱动咨询
November 9th, 2009
当别人求助你解决一个问题时，第一件要做的是什么？
不是挽起袖子解决问题。不是诊断症结在哪儿。甚至都不是问5个why。
首先要问的问题是：
- 如果这是一个问题，用什么数据能体现它？
- 如果问题被修复了，从这个数据上是否能反映？
然后，把度量放下去。每个度量项要有几个要素：
- 哪些数据？
- 如何得到数据？
- 以什么频率采集数据？
- 如何发布结果？
按照这几个要素制定一个跟踪度量方案。用这个方案先采集现有数据。根据现有数据定一个改进目标。
这就是你的──正在失败的──测试。先把测试放下去，然后再提任何改进措施。
Announcement: Fluorida 0.0.1
March 6th, 2008
I'm glad to announce that Fluorida 0.0.1 (the first preview release) is out. The project homepage is http://fluorida.googlecode.com
Fluorida is a Flex/Flash functional testing tool. It manipulates Flash just as real users do. It allows testers write test cases with simple but still expressive DSL. What makes Fluorida different is that it doesn't rely on mx.automation package, which is only available in Flex Builder Professional Edition. Other functional testing tools such as HP QuickTest Professional (formerly Mercury QuickTest Professional) and FunFX depend on mx.automation package but fortunately it seems like that dependency is not necessary to a Flex functional testing tool -- Fluorida is the evidence.
Version 0.0.1 is the first preview release. The main purpose is to gather feedbacks from the community. So far Fluorida has a basic "Tester" UI and it runs both locally and in web environment. It supports some fundamental actions. Check out the online demo and get started here.
What's next? It really depends on the feedback. We have a couple of things remaining on our TODO list, however the future direction is largely undecided yet. We are willing to see how people use it and figure out what to do in following releases. Therefore, your suggestion and feedback would be highly appreciated.
