Practical Measurements For Software Testing

Every software development company focusesInterfaces
on developing quality software. The only way toFunctional Capabilities
track the software quality isevaluating it at everyPerformance Levels
stage of its development. It requires some kindData Structures/Elements Safety
of metrics, which is obtained throughReliability
effectivetesting methods. Each stage of softwareSecurity/Privacy
testing is effectively monitored for the softwareQuality
QA.Constraints & limitations
1. Software measurements are used for:Next comes the updating of the crucial
2. Deriving basis for estimatesrequirement trace-ability matrix or RTM, which
3. Tracking project progressdetermines the number and types oftests.
4. Determining (relative) complexityWhile measuring the mapping of test cases, the
5. Understanding the stage of desired qualitynumber and priority of requirement it tests, its
6. Analyzing defectsexecution effort andrequirement coverage must
7. Validating best practices experimentallybe determined.
Here, some software testing metrics areThe Requirement compliance factor (RCF)
proposed for black box testing that has real worldmeasures the coverage provided by the test
applications. It discusses:cases to one or set of requirement(s).
Importance of software testing measurementMathematically,
Different techniques/processes for measuringRCFj=∑(Pi*Xi)
software testing(maxXi)*(∑Pi)i=1
Metrics for analyzing testingWhere,j is a set of requirements and (j=1-m);
Methods for measuring/computing the metricsXi=2, if the test case (say Tj) tests requirements
Advantages of implementing these metricsRi completely,
These metrics helps in understanding the=1, if it tests partially,
inadequacies in different software QA stages and=0, if otherwise.
finding better correctingpractices.Effectiveness=RCFj/Ej where Ej=Time required
What is measurement and why it is required?for executing a test case
The process of assigning numbers or symbols to4. Evaluating estimation accuracy
attributes of real world entities for describingRelative error=(A-E)/A where E is estimate of a
them according to definedrules is calledvalue and A is actual value.
measurement.For a collection of estimates, the mean RE for n
For developing quality software, severalprojects is
characteristics like requirements, time and effort,__ n
infrastructural cost,requirement testability, systemRE=1/n∑REii=1
faults, and improvements for more productiveFor a set of n projects, the mean magnitude of
resources should be measured.RE (MRE) is
Measuring software testing is required:___ n
1. If the available test cases cover all theMRE=1/n∑MREii=1
system's aspectsOf a set of n projects, an acceptable level for
2. For tracking problemsMRE is less than 0.25.
3. For quantifying testingIf K is the number of projects whose mean
Choose the suitable metricsmagnitude of relative error is less than or equal to
Several metrics can measure software-testingq,then the prediction quality pred(q)=K/n
process.5. Measurement of Efficiency in testing process
Here, the following types of metrics are identified:In software testing, we must keep tabs on what
Base metrics:we had planned and what we have actually
These raw data are collected in a testing effortachieved for measuring efficiency.
and applied in formulae used to derive CalculatedHere, the following attributes play major roles: -
Metrics.Cost: The Cost Variance (CV) factor measures
The Test Metrics comprise of the Number ofthe risk associated with cost.
Test Cases Passed, Failed, Under Investigation,CV=100*(AC - PC)/PC, AC=Actual Cost,
Blocked, Re-executed and Test Execution Time.PC=Planned/Budgeted Cost.
Calculated metrics:Effort: Effort Variance (EV) measures effort.
They convert the Base Metrics data into usefulEV=100*(AE - PE)/PE
information. Every test efforts must implement(AE=Actual Effort, PE=Planned Effort)
the following CalculatedSchedule: Schedule Variance (SV) is important for
Metrics:project scheduling.
% CompleteSV=100*(AD-PD)/PD where AD=Actual duration
% Defects Correctedand PD=Planned duration.
% Test CoverageCost of quality: It indicates the total effort
% Reworkexpended on prevention, appraisal and rework
% Test Cases Passed & Blockedfailure activities versus allproject activities.
% Test Effectiveness & EfficiencyPrevention Effort=Effort expended on planning,
% 1st Run Failurestraining and defect prevention.
% FailuresAppraisal Effort=Effort expended on quality
Defect Discovery Ratecontrol activities.
Defect Removal CostFailure effort=Effort expended on rework, idle
Measurements for Software Testingtime etc.
The corresponding software testing process inCOQ=100*(PE + AE + FE)/Total project effort.
software development measures each step forProduct -
ensuring quality product delivery.Size variance: It is the degree of variation
1. Software Size:between estimated and actual sizes.
The amount of functionality of an applicationSize Variance=100*(Actual Software Size-Initial
determines this and is calculated byEstimated Software Size)/Initial Estimated
Function Point AnalysisSoftware Size
Task Complexity Estimation MethodologyDefect density: It is the total number of defects
2. Requirements review:in software with respect to its size.
Before software development, the SoftwareDefect density=Total number of defects
requirement specifications (SRS) from the clientdetected/software size
are obtained. It must be:Mean Time Between Failures: MTBF is the mean
Completetime between two critical system failures or
Consistentbreakdowns.
CorrectMTBF=Total time of software system operation
StructuredNumber of critical software system failures.
RankedDefects: Defects are measured through:
TestableDefect distribution: It indicates the distribution of
Traceabletotal project defects.
UnambiguousDefect Distribution=100*Total number of defects
Validateattributed to the specific phase/Total number of
Verifieddefects.
The Review Efficiency is a metric that offersDefect removal effectiveness: Adding the number
insight on the review quality and testing.of defects removed during the phase to the
Review efficiency=100*Total number of defectsnumber of defects found laterapproximates this.
found by reviews/Total number of projectBenefits of implementing metrics in software
defectstesting:
3. Effectiveness of testing requirements:Improves project planning.
It is measured by maintaining RequirementUnderstanding the desired quality achieved.
Trace-ability matrix and specification ofHelps in improving the processes followed.
requirements, which should have:Analyzing the associated risks.
SRS Objective, purposeImproving defect removal efficiency.