Files and links (1)
CC BY V4.0, Open Access
Preprint
Large Language Models for Failure Mode Classification: An Investigation
ArXiv.org
Cold Spring Harbor Laboratory
2023
Abstract
In this paper we present the first investigation into the effectiveness of Large Language Models (LLMs) for Failure Mode Classification (FMC). FMC, the task of automatically labelling an observation with a corresponding failure mode code, is a critical task in the maintenance domain as it reduces the need for reliability engineers to spend their time manually analysing work orders. We detail our approach to prompt engineering to enable an LLM to predict the failure mode of a given observation using a restricted code list. We demonstrate that the performance of a GPT-3.5 model (F1=0.80) fine-tuned on annotated data is a significant improvement over a currently available text classification model (F1=0.60) trained on the same annotated data set. The fine-tuned model also outperforms the out-of-the box GPT-3.5 (F1=0.46). This investigation reinforces the need for high quality fine-tuning data sets for domain-specific tasks using LLMs.
Details
- Title
- Large Language Models for Failure Mode Classification: An Investigation
- Authors/Creators
- Michael StewartMelinda HodkiewiczSirui Li - Murdoch University, School of Information Technology
- Publication Details
- ArXiv.org
- Publisher
- Cold Spring Harbor Laboratory
- Identifiers
- 991005693764207891
- Murdoch Affiliation
- School of Information Technology
- Language
- English
- Resource Type
- Preprint
Metrics
5 File views/ downloads
60 Record Views