With the steep increase in the number of empirical studies and the advent of secondary studies that aggregate or perform meta-analysis on those studies, one main concern that arises is the quality of the primary studies (e.g., a single experiment). While researchers have raised concerns about different types of quality issues for many years, many of these issues remain, at least partially due to the complexity of empirical studies in our field—whether human subjects are involved or not. Specifically, problems related to appropriate design and the correct usage of statistical tests are crucial for the trustworthiness of the results. Consequently, as a community, we need to better understand these issues and how to mitigate, if not eliminate them. The workshop’s aim is to increase awareness of the importance of the quality and appropriateness of the methods applied during the different stages of software engineering empirical studies to yield reliable results and to disseminate this knowledge to the broader SE community.
Topics of interest include, but are not restricted to:
- Issues with the motivation and definition of appropriate research questions.
- Issues with the operationalization, e.g., missing standard metrics.
- Issues with the systematic identification of threats to validity and how to address them
- Issues selecting an appropriate study design and the consequences of that selection,
- Issues with drawing appropriate conclusions from the findings and generalizing them
- Issues with aggregating results from individual studies
- Issues with reproduction, replication, or families of studies
- Issues with quantitative, qualitative approaches and combined approaches
- Issues with understanding failures and successes: lessons learned
Paper categories and submission information
Papers must describe original work not submitted or presented at other forums (the studies could be). Accepted papers will be published in ICSE workshop proceedings.
Technical papers (6-8 pages): formally describe studies, explain how they were designed and executed to overcome challenges, highlight concrete results, and describe any advances to the state-of-the-art.
Experience reports (4-6 pages) describe the experience in conducting studies and lessons learned that address methodological issues.
Vision / Position papers (2-4 pages) describe new directions in conducting empirical studies including convincing arguments supported by clear rationale.
Submissions will be made via Easychair https://easychair.org/conferences/?conf=wsese2024.
The ICSE 2024 workshop proceedings will be prepared by IEEE CPS and published by ACM. All workshop papers must follow the ACM formatting instructions available at: Papers must be formatted according to ICSE 2024 guidelines (https://www.acm.org/publications/proceedings-template & the respective section here: submission process: https://conf.researchr.org/track/icse-2024/icse-2024-research-track). It is not possible to pay for extra pages.