2009 International Workshop on
Empirical Software Engineering in Practice (IWESEP 2009)
in Osaka, Japan, October 23, 2009

Online Proceedings are now available.

Advanced Program

9:00 - 9:15

Registration & Opening

9:15 - 10:15

Keynote Address

New Challenges in Software Measurement
Reiner R. Dumke (University of Magdeburg)
10:15 - 10:25

Break

10:25 - 11:15

Session 1 Software Verification and Validation

Measurement of IV&V activity effectiveness based on the experience of JAXA’s spacecraft project
Ryo Ujiie (JAXA)
Classification of Responsibility for Realization of Specifications of Requirements in Judgments in Software Developments in Japan
Kozo Kubo (Nara Institute of Science and Technology)
11:15 - 12:05

Session 2 Software Measurement

A Method to Analyze Crosscutting Features Based on Logical Coupling Sets of Product Release History
Kentaro Yoshimura (Hitachi)
MASU: Plug-in Platform for Metrics Measurement
Akira Saitoh (Osaka University)
12:05 - 13:50

Lunch

13:50 - 15:05

Session 3 Risk Assessment

Human Factors and Software Development Process
Konstantina Georgieva (University of Magdeburg)
The Effect of Normalization of Software Metrics in Fault-prone Module Prediction
Tatsuya Kuramoto (Nara Institute of Science and Technology)
An Empirical Study of Debugging with Breakpoints
Takuro Yoshimura (Nara Institute of Science and Technology)
15:05 - 15:35

Coffee Break & Research Demonstration

A Visualization System for Analyzing Large-Scale Complex Data
Akinori Ihara (Nara Institute of Science and Technology)
MASU: Plug-in Platform for Metrics Measurement
Goro Yamada (Osaka University)
Software-Tag Platform Tool for Visualizing Historical Data in Software Development
Tetsuya Okumura (Nara Institute of Science and Technology)
15:35 - 16:50

Session 4 Distributed System and Development

Next Generation E-Commerce
Robert Neumann (University of Magdeburg)
Qualitative Analysis of Real-time Distributed Systems Considering Network Congestion by Probabilistic Model Checker PRISM
Takeshi Nagaoka (Osaka University)
Analyzing Code Review Practices in OSS Communities: Are OSS Communities Really Open?
Shoji Fujita (Nara Institute of Science and Technology)
16:50 - 17:00

Closing

Titles and Abstracts

Keynote Address

New Challenges in Software Measurement

Presentator:
Reiner R. Dumke (University of Magdeburg)
Abstract:

Software measurement and metrics are key technologies to manage and to control software development projects. Measurement is essential for any engineering activity, by increasing the scientific and technical knowledge for both the practice of software development and for empirical research in software technology.

The goal is to identify process quality rules that cover the whole software process models and structures in order to achieve a quantitative software management and to identify open problems. We discuss a methodology achieving a holistic overview about quality-based relations between different components of the software development considering products, processes and resources.

Furthermore, the different standards, methodologies and approaches such as ISO 15939, e-Measurement and paradigm-related measurement and evaluation are discussed and described as basics for the empirical software engineering.

This presentation gives an overview about the current software measurement areas, problems, trends and scientific challenges. Examples of our Software Measurement Lab (SML@b) developments and results are described and further tasks on this field are discussed for future investigations and cooperations.

Session 1: Software Verification and Validation

Measurement of IV&V Activity Effectiveness Based on the Experience of JAXA’sSspacecraftProject

Presentator:
Ryo Ujiie (JAXA)
Abstract:

Japan Aerospace Exploration Agency (JAXA) has established and been implementing a Software Independent Verification and Validation (IV&V) process for spacecraft projects over ten years. The number of IV&V applied projects is increasing, while more critical functions are assigned to software, and the effectiveness of IV&V has been recognized In the planning activity, IV&V attributes and techniques are selected according to the system and software criticality and complexity. Although IV&V plan is characterized with the combination of attributes and techniques, it is difficult to measure the effectiveness of an attribute and technique.

We used to measure the effectiveness of the IV&V activity by analyzing our IV&V finding. It is understood from this measurement that the information of the findings was not enough to evaluate the effectiveness. In this study, we discuss the essential information, and establish the factors which should be recorded in our IV&V activity in order to measure the effectiveness.

Moreover, we suppose that nonconformance report is also available to evaluate the IV&V activity. IV&V finding is made by IV&V engineers, while nonconformance report is made by developers. The more objective measurement of IV&V activity effectiveness will be possible by analyzing nonconformance reports. We discuss how to analyze a nonconformance report in order to evaluate the effectiveness. In addition to nonconformance report, developer’s comment aboutIV&V activity will be also available. We also consider the comment to measure IV&V activity effectiveness.

Finally, we show the trial application of the new evaluation to some actual spacecraft projects. As a result of this trial application, the lessons and learned and future work will be given.

Classification of Responsibility for Realization of Specifications of Requirements in Judgments in Software Developments in Japan

Presentator:
Kozo Kubo (Nara Institute of Science and Technology)
Abstract:

It is important to distinguish whether a request, which a user proposes later, is a specification of a requirement, or an additional request for an alteration of the contract and for an additional payment in a software development. We can classify cases into five groups from some Japanese judgments, in which a vender must realize the user’s request, even though the user requests ambiguously, or he request after a definition was finished. When classification is done by these five points, a vender must realize the user’s request.

  1. By reviewing the requirement.
  2. By considering the objective of introduction of a system or the comparison with the previous system.
  3. By considering the common function of the software.
  4. By considering that a user cannot use the system without the function.
  5. By considering that a change or a modification of the request is slight.

A vender always must consider these five points and realize the user’s request, even if the request is not involved in the requirement. But on the contrary, if the user’s request is not involved in these five points, the user must make the additional payment. We expect this classification system is useful for the settlement of some disputes.

Session 2: Software Measurement

A Method to Analyze Crosscutting Features Based on Logical Coupling Sets of Product Release History

Presentator:
Kentaro Yoshimura (Hitachi)
Abstract:

This paper describes a method to analyze crosscutting features based on logical coupling of legacy product release history for migrating into software product line engineering. Crosscutting features help developer of large embedded systems to reduce the number of variable feature. However, time for analyzing and quantitative evaluations are problems to be solved.

The proposed method mines candidates for crosscutting features from the product release history, based on the logical coupling of software components. Also, the method applies the precision and the recall as metrics and determines the candidates quantitatively and automatically.

We apply the proposed method to an embedded software, automotive control system. The method extracted six crosscutting features with the evaluation metrics, 97% of precision and 31% of recall.

MASU: Plug-in Platform for Metrics Measurement

Presentator:
Akira Saitoh (Osaka University)
Abstract:

Code metrics measurement is a promising method for efficiently and effectively maintaining software systems. Measurement result objectively represents properties and states of the target system, and it is an indicator what we next should do for the system. Consequently, various code metrics have been proposed from now, and many research efforts addressed their usefulness.

However, it is not easy to apply code metrics measurement to actual maintenance process because source code analysis is mandatory to measure code metrics. There are many compiler-compiler tools to aid source code analysis. Unfortunately, their supports are up to syntax analysis, and complicated semantic analysis have to be implemented by hand. Moreover, when multiple metrics are required, we have to find measurement tools that can deal with each of the metrics. If there is no measurement tool that can handle with all of them, multiple measurement tools have to be used. However, their output formats will be different, so that we are complicated with handling them.

Therefore, we have developed a tool, MASU (Metrics Assessment plugin-platform for Software Unit of multiple programming languages). The most remarkable feature of MASU is that metrics measurement is completely separated from source code analysis. MASU analyzes source code and keeps the analyzed information in the internal database. User implemented logics of metrics measurement as plugins of MASU, the plugins obtain information necessary to measure metrics via API. MASU realizes that we measure various kinds of code metrics without implementing complicated source code analysis.

We have also developed Eclipse front-end to perform metrics measurement on Eclipse IDE. User can easily measure metrics on Eclipse. This front-end also shows difference of result we measured before.

Note:
The demonstration of this presentation is scheduled at Research Demonstration Session.

Session 3: Risk Assessment

Human Factors and Software Development Process

Presentator:
Konstantina Georgieva (University of Magdeburg)
Abstract:

The topic of my research work is about human factors and the risks that they introduce in the process of software development.

The presentation will begin with motivation why people are so important in the process of software development and why they are the key factor that has to be considered when we want to conduct risk management of the software system. Under software system we understand the union between people, software and hardware. With some examples it is shown the enormous complexity of the human being, which is underestimated and is not considered into details in the risk management methods.

A discussion of the most famous approaches for risk management is presented and it concludes with observing the main limitations in them. In my research work I adopt the well-known FMEA (Failure Mode and Effect Analysis) method and I want to extend it in a way that it evaluates the risks introduced from the people in the software lifecycle. The point to choose FMEA is first that it is a very famous approach in the mobile and engineering industry, but in the software industry is not so popular and second it gives very good analytical possibility because the main point in it is to break each process into its sub-processes until we reach the smallest step that we have to do. This is very important because we can observe the software process in its smallest detail and introduce the human risk factors in each of them.

I am discussing different human factors that take effect in the everyday working process and the presentation is concluded with future work, which summarizes the points to be completed in order that the proposed approach reaches its completeness and meaningfulness.

The Effect of Normalization of Software Metrics in Fault-prone Module Prediction

Presentator:
Tatsuya Kuramoto (Nara Institute of Science and Technology)
Abstract:

In software development, it is important to assign test effort to fault-prone modules, which are likely to include a bug, to ensure the reliability and not to waste test effort for fault-less modules. However, enough prediction accuracy is not always achieved when characteristics of a target project is different from that of a base project used for building a prediction model. This paper proposes a method to normalize software metrics used as predictor variables of the model, to get enough prediction accuracy for such a cross-project prediction. We evaluated the proposed method using data sets of nine projects from NASA IV & V Facility Metrics Data Program. As a result of the experiment, compared with a conventional method, the AUC value improved 0.07 on average.

An Empirical Study of Debugging with Breakpoints

Presentator:
Takuro Yoshimura (Nara Institute of Science and Technology)
Abstract:
Software debugging is a lengthy and tiresome part of software development since it is not very creative and it requires sort of patience in inspecting program code. However, debugging is also an inevitable part of development, as no programmer can write code without injecting any bugs. Unfortunately, while debugging performance greatly depends on the programmers' debugging skill, books on debugging techniques are rarely seen. Also, debugging is one of the most unfocused areas in software engineering, while there exists huge amount of studies on requirement analysis, design, programming, inspection, etc. This study focuses on the "debugger", which has a great potential to increase the debugging performance of programmers. Debuggers are tools that enable programmers to observe the execution of a program, stop it by "breakpoint" and inspect values in memory. Our goal is clarify debugging skills of effective use of debuggers, especially the breakpoint facility, which is most basic and important in using debuggers. To achive this goal, we used a measurement tool "Hackystat", which can record programmers' actions of using Eclipse debugging facility. We also build a tool to visualze and analze data recoreded by the Hackystat. We conducted a pilot experiment to analyze programmers' actions in debugging a small Java program (about 150 source lines of code) using Eclipse. As a result of analysis of 6 subjects' debugging actions, we found that expert programmers used more breakpoints than novices. Also, we found that novices tend to set breakpoints one by one to adjacent lines.

Research Demonstration

A Visualization System for Analyzing Large-Scale Complex Data

Presentator:
Akinori Ihara (Nara Institute of Science and Technology)
Abstract:

In this demonstration, we introduce “a visualization system for analyzing large-scale complex data”. This system has a 120-inch large display which is composed from four 65-inch high-definition LCD displays (full resolution: 1920×1080 pixels). The system can visualize several kinds of large-scale complex data at the same time. Using the system, we have been analyzing data of large-scale software development such as a number of source code, bug reports and history of developers’ activities, in order to reveal success and failure factors in a large software project at a fine grained level. An example of our studies is a social network analysis of communications among Apache developers. We collected data of the Apache developers’ mailing list, which includes over 101,000 messages exchanges among 11,000 developers for five years. Extracting post-reply relationships among the Apache developers from the mailing list data, we analyzed the communication structure of the Apache developers. The visualization system at this time allowed us to better understand the social structure in the Apache project by providing us fine-grained and large-sized visualizations of communications among the developers, though it was too difficult to visualize such the large-scale social network data with a common desktop PC and display. In this presentation, I would like to introduce the result of our analysis with the visualization system.

MASU: Plug-in Platform for Metrics Measurement

Presentator:
Goro Yamada (Osaka University)
Note:
The presentation of this demonstration is scheduled at Session2. Please see also that abstract.

Software-Tag Platform Tool for Visualizing Historical Data in Software Development

Presentator:
Tatsuya Okumura (Nara Institute of Science and Technology)
Abstract:

In the most software development cases, which Users leave development to Venders, there occur many misses such as design faults, scope creep or even more. "Software-Tag" has been developed as a technology to share software development data between Users and Vendors in an effort to prevent these software development mishaps. Software-Tag contains a great deal of information related to the development of a project. However, it is still hard to sort out what had occurred in the process of software development. Therefore, we need to focus on all kinds of phases and visualize Software-Tag.

In this demonstration, we introduce a visualization tool, which gives an image of the situation for Users to keep-up and check the progress of project, using Software-Tag. Other existing visualization tools are only capable of to be focused on lower-development process. With this tool, which covers up all the development process, which means the tool supports both lower and upper-development processes.

Session 4: Distributed System and Development

Next Generation E-Commerce

Presentator:
Robert Neumann (University of Magdeburg)
Abstract:

The introduction of E-Business concepts to enterprises has revolutionized the way in which corporate processes are driven and that to a degree that has never been seen before. By adding computer support to process execution and management, throughput and coordination could be enormously increased resulting into an overall strongly enhanced process efficiency.

New distribution channels and mechanism to incorporate customer touch points evolved in the course of the internet becoming a widely spread medium which was accessible by everybody. Nowadays, firms and retailers operate their own private websites and online shops, which enable customers to browse for product information and even order products they are interested in “over the net”. Over time, firms have made their virtual storefronts more and more attractive to customers, increased the information richness of their product descriptions and thus made the internet become the number one medium for conducting commercial transactions.

While the term E-Commerce and the traditional concept of online shops have been around for several decades by now, we believe that we have reached a point in time that allows questions for what comes next. Even though state-of-the-art E-Commerce has brought many advantages to our daily shopping experiences, there are still several open and unsolved problems that hinder virtual storefronts and online shops from becoming a medium that supports the human needs indivisibly and unobtrusively.

In this talk, we would like to first elaborate on the problems and shortcomings of the state-of-the-art in E-Commerce. In the second part, we will then describe our vision of moving from the current situation of complexity to a human-centered, customer-centered way (Normand, 1999) of conducting E-Commerce.

References
Normand, D. (1999). The Invisible Computer: Why Good Products Can Fail, The Personal Computer is so Complex, and Information Appliances are the Solution. The MIT Press.

Qualitative Analysis of Real-time Distributed Systems Considering Network Congestion by Probabilistic Model Checker PRISM

Presentator:
Takeshi Nagaoka (Osaka University)
Abstract:
The Internet is a best-effort network shared by a number of users, so there is no guarantee on the QoS properties such as network bandwidth, delay and throughput. Therefore, system developers preliminarily have to estimate the QoS by simulation techniques or mathematical analysis. Probabilistic model checking can evaluate performance, dependability and stability of information processing systems with random behaviors. PRISM is one of probabilistic model checkers. It handles automata with probabilities (discrete and continuous time Markov chains) and time elapse. This paper presents a discrete Markov chain for a real-time distributed system, and evaluates its QoS properties using the simulation function of PRISM. As the target real-time distributed system, we use an experimental system whose main subsystem is video data transmission which uses RTSP (Real Time Streaming Protocol) for the streaming protocol. The system has also ftp servers and clients which exploit tcp connections, as well as a packet generator that generates udp packets as background noise. Thus, the system involves several simultaneous connections; it may occur congestion. To validate the correctness of our model, we also model it in a model for the well-known network simulator NS--2. The paper gives the comparison of their simulation results. The comparison shows that the result of PRISM simulation is very similar to that of NS--2. The results show that the proposed method is useful to analyze the network performance.

Analyzing Code Review Practices in OSS Communities: Are OSS Communities Really Open?

Presentator:
Shoji Fujita (Nara Institute of Science and Technology)
Abstract:
Open source software (OSS) products are created by developers who voluntarily participate in the OSS projects.Certain products from OSS communities (e.g., Linux, Apache, PostgreSQL) favorably compare with proprietary software products and are prevalent in our daily life.A large number of OSS products and communities, however, result in failure or stagnation due to various factors.We suppose that a successful (surviving) community might change its form over time so that they can meet market (user) needs. That is, if the community need to provide high functional products, they are likely to accept contributions from new developers in a positive manner.In contrast, if the community needs to supply quality products, they would place much more value on contributions from core-developers than that from new developers.In order to observe such the difference of forms of an OSS community, in this presentation we focus on the review rate which is the proportion for reviewed patches to all the submitted patches.Especially, we analyze the differences between the review rate of patches from core developers and that from non-core developers.In doing so, we consider that we can capture the changes of the forms in OSS communities which would suggest insights on the success factor of OSS products and communities.In this presentation, we report results of our case study of the Apache HTTP Server community.