SEG3101 (Fall 2015) Non-Functional Requirements and Quality Measures

SEG3101 (Fall 2015) Non-Functional Requirements and Quality Measures

SEG3101 (Fall 2015) Non-Functional Requirements and Quality Measures Miguel Garzn, University of Ottawa Based on Powerpoint slides by Gunter Mussbacher (2009) with material from: Jo Atlee, Dan Berry (both University of Waterloo); R. Pressman; D. Damian; Amyot 2008, Som 2008 Table of Contents Non-Functional Requirements and Software Quality Attributes Software Quality Classifications of Non-Functional Requirements Quality Measures To measure is to know. If you cannot measure it, you cannot improve it.1 [1] Lord Kelvin (1824 - 1907) SEG3101 (Fall 2012). Non-Functional requirements 2 Introduction to Requirements Specification Software Quality

Classifications of NFRs Quality Measures Software Quality (1) Most definitions require compliance with requirements Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software.1 Implication: We need to be able to explicitly quantify requirements and verify that any solution meets them We need measures [1] Pressman, 1997 SEG3101 (Fall 2012). Non-Functional requirements 3 Introduction to Requirements Specification Software Quality Classifications of NFRs

Quality Measures Software Quality (2) An interesting phenomenon: Measurable objectives are usually achieved! Therefore, unless you have unrealistic values, requirements are usually met Important to know what measures exist! The chosen values, however, will have an impact on the amount of work during development as well as the number of alternatives and architectural designs from which developers may choose to meet the requirements SEG3101 (Fall 2012). Non-Functional requirements 4 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures

Types of Non-Functional Requirements (NFRs) Non-functional requir ements Product requir ements Ef ficiency requir ements Reliability requir ements Usability requirements Performance requirements Or ganizational requir ements Portability requirements Delivery requirements

Space requir ements External requirements Interoperability requirements Implementation requir ements Ethical requirements Standards requirements Legislative requirements Privacy requirements Safety requirements

Source: Gerald Kotonya and Ian Sommerville, Requirements Engineering Processes and Techniques, Wiley, 1998 SEG3101 (Fall 2012). Non-Functional requirements 5 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Interesting Resources on Wikipedia ISO/IEC 9126 Software engineering Product quality http://en.wikipedia.org/wiki/ISO/IEC_9126 "ilities" http://en.wikipedia.org/wiki/Ilities Software Quality, and how to measure it http://en.wikipedia.org/wiki/Software_quality SEG3101 (Fall 2012). Non-Functional requirements 6

Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures ISO/IEC 9126 Qualities Nowadays, more and more frequently replaced by: ISO/IEC 25000:2005 Software Engineering -- Software product Quality Requirements and Evaluation (SQuaRE) -- Guide to SQuaRE SEG3101 (Fall 2012). Non-Functional requirements 7 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Quantification Non-functional requirements need to be measurable

Avoid subjective characterization: good, optimal, better... Values are not just randomly specified Must have a rationale Stakeholder must understand trade-offs Important to rank and prioritize the requirements Precise numbers are unlikely to be known at the beginning of the requirement process Do not slow down your initial elicitation process Ensure that quality attributes are identified Negotiate precise values later during the process SEG3101 (Fall 2012). Non-Functional requirements 8 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Measures vs. Metrics We use measures in a generic way but there is actually a

distinction between measures and metrics For example, consider availability: Metric: mean time between failures Measure: 23 days between failures (a specific observation!) SEG3101 (Fall 2012). Non-Functional requirements 9 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Performance Measures (1) Lots of measures Response time, number of events processed/denied in some interval of time, throughput, capacity, usage ratio, jitter, loss of information, latency... Usually with probabilities, confidence interval Can be modeled and simulated (mainly at the architectural level) performance prediction

Queueing model (LQN), process algebra, stochastic Petri nets Arrival rates, distributions of service requests Sensitivity analysis, scalability analysis SEG3101 (Fall 2012). Non-Functional requirements 10 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Performance Measures (2) Examples of performance requirements The system shall be able to process 100 payment transactions per second in peak load. In standard workload, the CPU usage shall be less than 50%. Production of a simple report shall take less than 20 seconds for 95% of the cases. Scrolling one page up or down in a 200 page document shall take at most 1 second.

SEG3101 (Fall 2012). Non-Functional requirements 11 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Reliability Measures (1) Measure of the degree to which the system performs during a request Ability to perform a required function under stated conditions for a specified period of time Very important for critical, continuous, or scientific systems Can be measured using Defect rate Degree of precision for computations SEG3101 (Fall 2012). Non-Functional requirements 12

Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Reliability Measures (2) Examples The precision of calculations shall be at least 1/106. The system defect rate shall be less than 1 failure per 1000 hours of operation. SEG3101 (Fall 2012). Non-Functional requirements 13 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures

Availability Measures (1) Definition: Percentage of time that the system is up and running correctly Probability that the system is up and running when needed! Can be calculated based on Mean-Time to Failure (MTBF) and Mean-Time to Repair (MTTR) MTBF : Length of time between failures MTTR : Length of time needed to resume operation after a failure Availability = MTBF/(MTBF+MTTR) May lead to architectural requirements Redundant components (lower MTBF) Modifiability of components (lower MTTR) Special types of components (e.g., self-diagnostic) Modeling/Estimating availability e.g. Markov models SEG3101 (Fall 2012). Non-Functional requirements 14 Introduction to Requirements Specification Software Quality

Classifications of NFRs Quality Measures Availability Measures (2) Examples The system shall meet or exceed 99.99% uptime. The system shall not be unavailable more than 1 hour per 1000 hours of operation. Less than 20 seconds shall be needed to restart the system after a failure 95% of the time. (This is a MTTR requirement) Availability Downtime 90% 99% 36.5 days/year 99.9% 99.99% 8.76 hours/year

3.65 days/year 52 minutes/year 99.999% 5 minutes/year 99.9999% 31 seconds/year SEG3101 (Fall 2012). Non-Functional requirements 15 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Security Measures (1) There are at least two measures: 1. The ability to resist unauthorized attempts at usage 2. Continue providing service to legitimate users while under denial of service attack (resistance to DoS attacks) Measurement methods:

Success rate in authentication Resistance to known attacks (to be enumerated) Time/efforts/resources needed to find a key (probability of finding the key) Probability/time/resources to detect an attack Percentage of useful services still available during an attack Percentage of successful attacks Lifespan of a password, of a session

Encryption level SEG3101 (Fall 2012). Non-Functional requirements 16 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Security Measures (2) May lead to architectural requirements Authentication, authorization, audit Detection mechanisms Firewall, encrypted communication channels Can also be modeled (logic ...) Examples of requirements The application shall identify all of its client applications before allowing them to use its capabilities. At least 99% of intrusions shall be detected within 10 seconds.

SEG3101 (Fall 2012). Non-Functional requirements 17 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Usability Measures (1) In general, concerns ease of use and of training end users. The following more specific measures can be identified: Learnability Proportion of functionalities or tasks mastered after a given training time Efficiency Acceptable response time Number of tasks performed or problems resolved in a given time Number of mouse clicks needed to get to information or functionality Memorability Number (or ratio) of learned tasks that can still be performed after not using the system for a given time period

Error avoidance Number of error per time period and user class Number of calls to user support SEG3101 (Fall 2012). Non-Functional requirements 18 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Usability Measures (2) Error handling Mean time to recover from an error and be able to continue the task User satisfaction Satisfaction ratio per user class Usage ratio Examples The system should enable at least 80% of users to book a guest within 5 minutes after a 2-hour introduction to the system.

The system shall enable novice users to perform tasks X and Y in less than 15 minutes. The system shall enable expert users to perform tasks X and Y in less than 2 minutes. The satisfaction level of the system shall be very good or better for at least 80% of customers polled after a 3 months usage period . SEG3101 (Fall 2012). Non-Functional requirements 19 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Maintainability Measures (1) Measures ability to make changes quickly and cost effectively Extension with new functionality Deleting unwanted capabilities Adaptation to new operating environments (portability) Restructuring (rationalizing, modularizing, optimizing, creating reusable components) Can be measured in terms of

Coupling/cohesion metrics, number of anti-patterns, cyclomatic complexity Mean time to fix a defect, mean time to add new functionality Quality/quantity of documentation Measurement tools code analysis tools such as IBM Structural Analysis for Java (http://www.alphaworks.ibm.com/tech/sa4j) SEG3101 (Fall 2012). Non-Functional requirements 20 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Maintainability Measures (2) Examples of requirements Every program module must be assessed for maintainability according to procedure XYZ. 70% must obtain highly maintainable and none poor. The cyclomatic complexity of code must not exceed 7.

No method in any class may exceed 200 lines of code. Installation of a new version shall leave all database contents and all personal settings unchanged. The product shall provide facilities for tracing any database field to places where it is used. SEG3101 (Fall 2012). Non-Functional requirements 21 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Testability Measures Measures the ability to detect, isolate, and fix defects Time to run tests Time to setup testing environment (development and execution) Probability of visible failure in presence of a defect Test coverage (requirements coverage, code coverage) May lead to architectural requirements Mechanisms for monitoring Access points and additional control

Examples The delivered system shall include unit tests that ensure 100% branch coverage. Development shall use regression tests allowing for full retesting in less than 12 hours. SEG3101 (Fall 2012). Non-Functional requirements 22 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Portability Measures Measure ability of the system to run under different computing environments Hardware, software, OS, languages, versions, combination of these Can be measured as Number/Enumeration of targeted platforms (hardware, OS) Proportion of platform specific components or functionality Mean time to port to a different platform

Examples No more than 5% of the system implementation shall be specific to the operating system. SEG3101 (Fall 2012). Non-Functional requirements 23 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Integrability and Reusability Measures Integrability Measures ability to make separated components work together Can be expressed as Mean time to integrate with a new interfacing system Reusability Measures ability that existing components can be reused in new applications Can be expressed as Percentage of reused requirements, design elements, code, tests Coupling of components

Degree of use of frameworks SEG3101 (Fall 2012). Non-Functional requirements 24 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Robustness Measures Measure ability to cope with the unexpected Percentage of failures on invalid inputs Degree of service degradation Minimum performance under extreme loads Active services in presence of faults Length of time for which system is required to manage stress conditions Examples The estimated loss of data in case of a disk crash shall be less than 0.01%. The system shall be able to handle up to 10000 concurrent users when satisfying all their requirements and up to 25000 concurrent users with browsing capabilities.

SEG3101 (Fall 2012). Non-Functional requirements 25 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Domain-specific Measures The most appropriate quality measures may vary from one application domain to another, e.g.: Performance Web-based system: Number of requests processed per second Video games: Number of 3D images per second Accessibility Web-based system: Compliance with standards for the blind Video games: Compliance with age/content ratings systems (e.g., no violence)

SEG3101 (Fall 2012). Non-Functional requirements 26 Introduction to Requirements Specification Software Quality Classifications of NFRs Quality Measures Other Non-Functional Requirements What about NFRs such as fun or cool or beautiful or exciting? How can these be measured? The lists of existing quality attributes are interesting but they do not include all NFRs. It is sometimes better to let customers do their brainstorming before proposing the conventional NFR categories. In any case, we must also refine those goals into measurable requirements. SEG3101 (Fall 2012). Non-Functional requirements

27 Dilbert on Quality SEG3101 (Fall 2012). Non-Functional requirements 28

Recently Viewed Presentations

  • Climate Variability and Change in the Antarctic Peninsula

    Climate Variability and Change in the Antarctic Peninsula

    Antarctic Climate Change and the Environment John Turner British Antarctic Survey
  • Spracovanie hudobných fondov ŠVK v Banskej Bystrici vo ...

    Spracovanie hudobných fondov ŠVK v Banskej Bystrici vo ...

    Epoka e-czytelnictwa. Bibliotekarstwo w obliczu nowych wyzwań OPOLE, 6. 3. 2014
  • Economic goal 3: Low inflation - Weebly

    Economic goal 3: Low inflation - Weebly

    Economic Goals. Low Inflation. ... Economic goal Low inflation. A Definition of Inflation. Prices of goods and services will be changing on a regular basis in a market capitalist economy. Some prices of goods or services fall over time for...
  • Port Bienville Rail EIS Scoping Meeting Presented by:

    Port Bienville Rail EIS Scoping Meeting Presented by:

    Project History. HCPHC secured USDOT Grant for Port Bienville Rail Study in 2007. FRA is led Federal Agency overseeing the EIS. Mississippi Department of Transportation is Contracting Agency and manages the Study. Consultant team is led by CDM Smith ....
  • Science, Intelligent Design and Evolution

    Science, Intelligent Design and Evolution

    The axis, or hinge, of the fold is the line of sharpest curvature of any given layer. Wherever we measure the strike and dip of beds on such a fold, we find that the attitudes are always parallel to a...
  • Ashtrax - Providence University

    Ashtrax - Providence University

    UBrowser is an open source test mule that renders interactive web pages onto geometry using OpenGL® and an embedded instance of Gecko, the Mozilla rendering engine. 群體(groups):2人以上之人員,執行某項工作 群體決策制定之本質:雖然多數企業是階層式的,但是 ...
  • Solving Linear Equations in Two Variables

    Solving Linear Equations in Two Variables

    Planning a Joint Method Take turns to explain your method and how your work could be improved. Listen carefully to each other. Ask questions if you don't understand. When everyone in the group has explained their method, plan a joint...
  • PowerPoint Presentation

    PowerPoint Presentation

    "The benefits of paid sick leave extend far beyond the positive impact on individual families. It's also about making our businesses run better, and protecting the health and welfare of their customers."