A New Perspective and Progressive Applications of the Large Language Models in Geotechnical Engineering Field

RS-GPT (Rock and Soil Generative Pre-Trained Transformers): Large Language Model solution for geotechnical engineering applications.  Seok Hyeon Chai[1], Jason Huang [2], Sina Javankhoshdel [3], and Thamer Yacoub[4] 1,2,3,4 Rocscience, Toronto, Canada steve.chai@rocscience.com The use of Natural Language Processing (NLP) in the field of engineering seems complex with many applications which require calculations and engineering judgements. Large Language Models (LLM) have been successfully presented to be one of the most effective solutions in many specific fields such as arts and literature, to finance, to even software development. However, the use of LLM in the field of Civil Engineering and specifically in Geotechnical Engineering has not been discussed often compared to the other fields of expertise that utilize the functionalities of Application Programming Interfaces (API) of OpenAI. This paper aims to cover the details of the framework and breakdown of how LLMs can be used to train a domain-specific dataset, specifically related to settlement analysis, ground improvement, to slope stability analysis and provide results that are generated from a fine-tuned model with the dataset of research papers and technical documents provided by geotechnical engineers. The paper presents three folds: introduction for LLM models with pre-trained models, and choice of the models used. Then, the paper discusses the fine-tuning process of the LLM model with a domain-specific database. The paper provides benchmarks and metrics for performance of the model using the technique called Recall-Oriented Understudy for Gisting Evaluation (ROGUE) for validation of the outputs by comparing the result with prompt-completion pairs.  With sufficient data for the prompt-completion pairs from the experts in the field and also from the scientific literature and localized technical documents, the LLM model trained with data related to the geotechnical engineering domain provides reasonable responses that otherwise would not have been achieved with the original ChatGPT LLM model. This paper introduces innovative way to manage current database of geotechnical documents and literature that can be optimized to be used with LLMs.  The paper provides discussion of the comparative results and the future potential capabilities of its fine-tuned model.  Key words: Large Language Models (LLM), ChatGPT, Civil Engineering, settlement analysis, slope stability analysis, fine tuned models, APIs with OpenAI.