For the first time, ICSOC 2025 will have a reproducibility verification based on artifact submissions. This is to promote rigorous evaluation and sharing of data, code, and methodologies to enhance the credibility and impact of scientific contributions. Research reproducibility is essential for scientific integrity and ensures that the reported results are reliable and can be recreated, verified and built upon by others.

Authors of accepted main track papers are invited to submit a companion artifact description paper to the special Artifact Evaluation track.

Authors should share their research artifacts, such as datasets, source code, and supplementary materials, via reputable and persistent public repositories such as Zenodo, Figshare, GitHub, or institutional archives. Such public repositories offer stable, long-term access to research outputs, providing a reliable source for references and citations. To support open science, it is advised (but not required) that these artifacts be released under open data licenses, allowing for broad use and redistribution. On the other hand, cloud-based services like Dropbox and Google Docs are not accepted due to their volatile links, which can expire or change, causing broken references and unreliable access. Additionally, these cloud services may lack robust metadata and versioning features. Therefore, authors should opt for dedicated repositories to ensure data preservation and consistent availability.

The Artifact Evaluation paper should contain clear guidelines on how the artifacts should be assessed. For executable artifacts (e.g., code), instructions on how to run them and how to interpret the results should be provided. The use of executable containers is encouraged. For data artifacts, a clear description (metadata) and information on the origins and provenance of the data must be given. For other type of artifacts (e.g., model specifications, standards), the authors should exercise their judgement in providing the information necessary to assess the validity of the respective artifacts.

The artifact submissions will be evaluated based on the clarity of the instructions and the ability of the Artifact Evaluation Committee to verify the results reported in the original ICSOC paper.

Important dates

Paper instructions

• Papers should be formatted according to Springer's LNCS Formatting Guidelines.

• Submissions must be in English and must not exceed 6 pages.

• The title of the paper MUST be "Artifact for '{Original title of the main ICSOC paper}'"

• In the paper, the authors should provide the following information:

  • A link to a public repository (e.g., GitHub, Zenodo) containing the artifact(s). A DOI or permanent link is encouraged.

  • Clear documentation and a step-by-step guide on how to reproduce the results reported in the paper.

  • (When applicable) A description of the experimental setup, including hardware and software dependencies. Instructions to run the code. Code containers are encouraged.

  • (When applicable) A description of the data origins and methods of data collection.

  • Licensing information to clarify usage rights.

  • A clear statement of any additional requirements (e.g., cloud service accounts, specific hardware) that may prevent reproducibility. When the work relies on cloud services, authors should provide instructions on how reviewers can replicate the environment at minimal cost (e.g., locally or free-tier services). If replicating the full original environment is practically infeasible, authors are encouraged to provide a simplified setup that allows to verify the basic executability and integrity of the code.

Submission instructions

All artifact papers must be submitted electronically to the Conference Submission System. The "Artifact Evaluation" track should be selected during submission.

Submission link: https://easychair.org/conferences/?conf=icsoc2025

The evaluation will be single-blind, and the Artifact Evaluation Committee will have access to the camera-ready version of the accepted main paper. This means there is no need to anonymize the artifact!

Contact information

For any questions regarding the Artifact Evaluation track, please contact the Reproducibility Chair, Victoria Degeler <v.o.degeler@uva.nl>.